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Preface 


To  "fly  and  fight"  effectively,  the  Air  Force  relies  on 
modem  weapon  systems  which  are  expensive  and  complex.  These 
systems  make  extensive  use  of  computer  hardware  and  software 
to  perform  many  functions  which  were  previously  performed 
manually,  by  hardware,  or  were  not  able  to  be  performed  at 
all  prior  to  the  advent  of  computer  technology.  As  a  re¬ 
sult,  management  of  the  development  and  acquisition  '*f  soft¬ 
ware  is  a  subject  of  increasing  importance  to  the  Air  Force. 

This  subject  is  also  important  to  me  for  two  reasons. 
First,  I  experienced  many  of  the  frustrations  and  disappoint¬ 
ments  normally  associated  with  software  acquisition  as  a 
member  of  a  major  command,  control,  and  communications (C^) 
system  program  office  in  my  last  assignment.  Secondly,  my 
follow-on  assignment  is  to  a  Computer  Resources  Acquisition 
Management  Study  Group  at  Headquarters  Air  Force.  Hopefully, 
the  results  of  this  research  will,  in  some  small  way,  be 
helpful  in  solving  the  "software  problem. " 

There  are  many  people  whose  help  and  contributions  made 
this  research  effort  possible.  I  offer  my  sincerest  thanks 
to  all  of  them.  I  am  especially  grateful  to  all  those  "soft¬ 
ware  experts"  who  assisted  in  collecting  the  research  data. 

A  special  word  of  appreciation  is  extended  to  Mr.  Charles  £. 
Bobbish  for  his  invaluable  insight  and  advice  during  the  past 
year. 

I  also  express  my  deep  appreciation  and  gratitude  to  my 
advisor,  Professor  Charles  W.  McNichols  for  his  advice  and 
encouragement  throughout  the  research  period.  In  addition, 
thanks  are  extended  to  my  reader,  Professor  Alan  A.  Ross, 
for  his  invaluable  assistance.  This  study  would  not  have 
been  possible  without  their  help. 

Most  of  all,  I  am  deeply  indebted  to  the  Brooks  family. 
Susan  patiently  strived  for  perfection  while  typing  this 


ii 


thesis  at  all  hours  of  the  day»  and  Terry  reformatted  several 
tables  and  figures  in  order  to  present  the  results  more 
clearly. 

Finally,  I  express  my  undying  warmth  and  affection  to 
those  special  friends  who  provided  moral  support  and  encour¬ 
agement  during  the  darkest  moments  of  the  past  16  months. 


Virgil  L.  "lee'’  Cooper 
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Abstract 

The  importance  of  computer  hardware  and  software  in  the 
Department  of  Defense (DoD)  has  increased  over  the  past  20 
years  to  the  point  where  computer  technology  is  vital  to  the 
defense  of  our  country.  In  addition,  this  technology  has 
placed  a  tremendous  strain  on  the  fiscal  assets  of  the  DoD 
and  Air  Force (AF). 

This  thesis  documents  an  investigation  of  the  software 
requirements  allocation  process  in  the  acquisition  and  man¬ 
agement  of  a  major  defense  system.  More  specifically,  this 
research  identified  the  DoD  and  AF  policies  and  procedures 
on  CPCI  selection,  determined  what  criteria  are  currently 
used  by  system  program  off ices (SPOs)  and  contractors  to  al¬ 
locate  system  requirements  to  CPC  Is,  and  evaluated  the  fea¬ 
sibility  and  potential  impacts  of  an  alternate  approach  to 
CPCI  selection,  called  "horizontal  allocation. " 

An  extensive  literature  review  was  performed,  and  a 
semi-structured  interview  questionnaire  was  executed  to  col¬ 
lect  the  research  data.  The  questionnaire  was  designed  to 
capture  both  objective  and  subjective  data  from  a  sample 
population  that  included  45  "software  experts"  from  10  AF 
organizations,  2  Federal  Contract  Research  Centers,  and  11 
Aerospace  contractors.  Both  qualitative  and  quantitative 
analysis  techniques  were  used  to  interpret  the  interview 
results.  The  qualitative  analysis  consisted  of  categorizing, 
analyzing,  and  summarizing  the  answers  expressed  by  the  re¬ 
spondents.  Three  statistical  techniques  (Pearson  Product- 
Moment  Correlation,  Chi-square  test,  and  Student's  t  test) 
were  applied  to  the  quantitative  data. 

The  results  should  lead  to  a  better  understanding  of  the 
CPCI  selection  process.  DoD  and  AF  policies  on  CPCI  selec¬ 
tion  require  that  computer  resources  be  managed  as  items  of 
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major  importance  during  the  system  acquisition  life  cycle; 
and  that  computer  hardware  and  software  be  specified  and 
treated  as  conf iguration  items.  On  most  programs  evaluated, 
functional  modularity  and  previous  experience  were  the  pri¬ 
mary  considerations  used  to  allocate  system  requirements  into 
CPCIs.  This  resulted  in  significant  problems  with  assessing 
development  status,  achieving  system  integration,  completing 
meaningful  tests,  and  documenting  the  system.  On  the  other 
hand,  the  selection  of  CPCIs  on  the  basis  of  horizontal  al¬ 
location  is  feasible  and  promises  to  favorably  impact  the 
cost,  schedule,  performance,  and  management  parameters  nor¬ 
mally  associated  with  an  acquisition  program.  This  alternate 
approach  is  based  on  defining  a  software-intensive  system  in 
terms  of  system  versions  or  models,  each  of  which  contains 
end-use  system  functional  capabilities. 


AN  INVESTIGATION  OF  THE  SOFTWARE  REQUIREMENTS 
ALLOCATION  PROCESS  IN  THE  ACQUISITION  AND 
MANAGEMENT  OF  A  MAJOR  DEFENSE  SYSTEM 


Modem  weapon  systems1  make  extensive  use  of  computers 
and  software  to  perform  functions  which  were  previously  per¬ 
formed  manually,  by  hardware,  or  were  not  performed  at  all 
prior  to  the  advent  of  computer  technology.  Ab  a  result, 
the  importance  of  software  in  the  Air  Force (AF)  continues 
to  intensify  as  new  systems  emerge  in  response  to  increasing 
threats  and  declining  force  levels.  Thus,  computer  tech¬ 
nology  has  become  central  to  the  Air  Force's  ability  to  per¬ 
form  its  role  and  mission. 

This  technology  has  placed  a  tremendous  strain  on  the 
fiscal  assets  of  the  Department  of  Defense (DoD)  and  the  AF. 
In  197^,  estimates  of  the  annual  Automatic  Data  Processing 
(ADP)  costs  in  the  DoD  were  $2.9  billion  to  $3.6  billion  for 
software  and  a  total  of  $6.2  to  $8.3  billion  when  hardware 

1  In  this  thesis,  the  term  weapon  system  refers  to  any 
DoD  system  that  is  not  of  a  general  purpose,  commercially 
available  nature.  That  is,  it  includes  aircraft  systems i 
space  and  missile  systems i  command,  control,  and  communica¬ 
tions  (C^)  systems i  and  intelligence  systems.  A  glossary  of 
definitions  for  common  terms  used  throughout  this  thesis  is 
included  as  Appendix  A. 

2 

A  listing  of  the  acronyms  and  abbreviations  used  in 
this  thesis  is  attached  as  Appendix  B. 
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and  ADP  resources  were  included  (Ascht  Kelliher,  Locher,  and 
Connors,  1975  b 13-16),  The  software -hardware  cost  ratio  was 
approximately  1«4  in  1955,  while  the  1985  ratio  is  projected 
to  be  9«1  (Asch,  ,  1975  bO-48).  In  fact,  on  one 

existing  program,  the  World  Wide  Military  Command  and  Control 
System (WWMCCS ) ,  the  ratio  is  already  in  that  vicinity 
(Myers,  1978i13).  Given  such  ratios,  it  becomes  increasingly 
important  not  only  to  cope  with  the  technology  but  to  master 
and  exploit  it,  both  from  a  technical  point  of  view  and, 
equally  important,  from  a  management  and  policy  one. 

The  need  to  manage  software  as  a  critical  component  of 
a  defense  system  over  its  life  cycle  is  becoming  widely  rec¬ 
ognized.  A  general  awareness  of  this  need  as  a  defense-wide 
problem  has  been  growing  as  software  problems  have  reached 
top-level  management  visibility  with  increasing  regularity. 

In  July  1978,  Barry  C.  DeRoze  and  Thomas  H.  Nyman  of  the 
Office  of  Secretary  of  Defense  stated,  "It  is  our  opinion 
that  DoD  has  been  doing  a  poor  job  managing  this  increasingly 
Important  resource,  and  further,  we  have  been  doing  little 
to  encourage  the  application  of  science  and  technology  to 
improve  it.  Both  of  these  shortcomings  must  change I" 

(DeRoze  and  Nyman,  1978«309). 

Background 

In  reflecting  on  a  number  of  large,  complex  computer 
systems,  Dr.  Frederick  Brooks  of  the  University  of  North 
Carolina,  described  the  development  of  large  software 
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projects  as ,  "Like  the  mortal  struggle  of  prehistoric  beasts 

trying  to  escape  the  tar  pitsi 

Large  systems  programming  has  over  the 
past  decade  been  such  a  tar  pit,  and 
many  great  and  powerful  beasts  have 
thrashed  violently  in  it.  Most  have 
met  goals,  schedules,  and  budgets. 

Large  and  small,  massive  and  wiry, 
team  after  team  has  become  entangled 
in  the  tar.  No  one  thing  seems  to 
cause  the  diff iculty-any  particular 
paw  can  be  pulled  away.  But  the  ac¬ 
cumulation  of  simultaneous  and  inter¬ 
acting  factors  bring  slower  and  slower 
motion.  Everyone  seems  to  have  been 
surprised  by  the  stickiness  of  the 
problem,  and  it  is  hard  to  discern  the 
nature  of  it.  But  we  must  try  to  under¬ 
stand  it  if  we  are  to  solve  it"  (Brooks, 

1975**0. 

For  many  years,  the  DoD  has  had  its  own  version  of  the 
tar-pit  problem.  Difficulties  have  appeared  in  systems  sup¬ 
porting  all  functional  areas,  including  management  support, 
command  and  control,  intelligence,  communications,  and  em¬ 
bedded  computer  systems.  In  fact,  recent  defense-sponsored 
studies  have  shown  that  most  software  development  projects 
are  unsuccessful  in  terms  of  performance,  schedule,  and  cost. 
The  final  software  product  delivered  to  the  user  often  devi¬ 
ates  significantly  from  the  original  specification!  delivery 
schedules  are  sometimes  slipped  for  years i  and  cost  overruns 
are  often  on  the  order  of  200  to  300  percent  (Ramamsorthy, 
1975*^6).  Thus,  one  of  the  most  critical  periods  of  the 
software  life  cycle  which  needs  additional  management  empha¬ 
sis  is  the  acquisition  process. 

The  DoD  policy  on  major  system  acquisition  states, 


"Responsibility  for  the  management  of  system  acquisition 
programs  shall  be  decentralized  to  the  DoD  components  except 
for  the  decisions  retained  by  the  Secretary  of  Defense" 

(DoDD  5000.1,  1977 «2).  Thus,  DoD  components  are  responsible 
for  a  continuing  analysis  of  their  missions  to  identify  mis¬ 
sion  needs  and  to  define,  develop,  produce,  and  deploy  sys¬ 
tems  to  satisfy  those  needs.  The  system  acquisition  process, 
then,  is  a  sequence  of  specified  phases  of  program  activity 
and  decision  points  directed  at  the  achievement  of  estab¬ 
lished  program  objectives. 

The  process  is  initiated  with  the  approval  of  a  mission 
need  (DSARC  0)  and  extends  through  five  major  phases  with  an 
affirmative  decision  required  as  preliminary  to  proceeding 
into  the  second,  third,  and  fourth  phases  (Ref.  Figure  l). 

Thus,  the  system  acquisition  process  is  defined  as 

a.  the  conceptual  phase,  followed  by  a  DoD  program 
decision  (DSARC  l), 

b.  the  validation  phase,  followed  by  a  DoD  ratifica¬ 
tion  decision  (DSARC  2), 

c.  the  full-scale  development  phase,  followed  by  a  DoD 
production  decision  (DSARC  3)» 

d.  the  production  phase,  and 

e.  the  deployment  phase  (AFSCP  800-3,  1 976 1 1 —1 ) . 

The  implementation  of  this  process  in  the  Air  Force  is 
normally  delegated  to  the  Air  Force  Systems  Command (AFSC ) . 

Program  Offices (POs)  are  then  established  in  one  of  the  pro¬ 
duct  divisions  (i.e..  Electronic  Systems  Division,  Aeronau¬ 
tical  Systems  Division,  or  Space  and  Missile  Systems 
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Figure  1.  Systems  Acquisition  Life  Cycle  Phases 


Organization)  to  perform  the  acquisition  management  function. 
This  is  accomplished  by  a  complex  set  of  interrelated,  but 
separately  identified,  management  disciplines.  These  disci¬ 
plines  are  engineering,  procurement,  program  control,  test 
and  evaluation,  deployment,  integrated  logistics  support, 
data  management,  and  conf iguration  management. 

These  disciplines  represent  separate  areas  of  management 
responsibility  which  correspond,  largely,  with  individual 
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career  specialities  of  PO  personnel.  They  are  also  the  basis 
for  the  classical  PO  organization  and  for  major  topics  ad¬ 
dressed  in  the  various  acquisition  management  regulations, 
specifications,  and  standards.  However,  two  of  these  disci¬ 
plines,  engineering  and  configuration  management,  are 

5 


intimately  related  in  the  system  acquisition  process.  En¬ 
gineering,  as  used  here,  refers  to  any  or  all  of  the  various 
areas  of  technical  effort  involved  in  acquiring  a  DoD  system. 
It  is  the  general  term  which  encompasses  system  engineering, 
equipment  engineering,  software  engineering,  and  human  fac¬ 
tors  engineering.  Within  this  broad  concept,  the  engineering 
discipline  is  responsible  for  requirements  analysis,  design 
studies,  evaluation  of  performance,  cost  and  schedule  im¬ 
pacts  of  engineering  change  proposals (ECPs ) ,  deviations,  and 
waivers,  and  technical  direction  to  the  developer.  The  pro¬ 
ducts  of  these  analyses,  studies,  and  evaluations  are  the 
concern  of  configuration  management. 

Therefore,  configuration  management  is  a  discipline  that 
integrates  the  technical  and  administrative  direction  and 
surveillance  toi 

a.  identify  and  document  the  functional  and  physical 
characteristics  of  a  configuration  item, 

b.  control  changes  to  those  characteristics,  and 

c.  record  and  report  change  processing  and  implementa¬ 
tion  status  (AFR  65-3,  1975 «A2). 

Thus,  this  discipline  is  categorized  into  three  major  areas 
of  effort i  identification,  control,  and  status  accounting. 
The  first  of  these,  configuration  identification,  is  estab¬ 
lished  in  the  form  of  technical  documentation  that  becomes 
more  detailed  as  development  proceeds  through  the  acquisition 
cycle.  Three  stages  of  configuration  identification  are 
generally  employed  during  the  development  of  a  system.  They 


are  i 
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(1) 


Functional  Configuration  Identification ( FC I ) . 

First,  the  FCI  for  the  system  is  defined  in  the 
system  specification  or  system-segment  specifica¬ 
tion.  Specifically,  the  specification  prescribes 
all  necessary  functional  characteristics,  required 
performance  parameters,  the  tests  required  to  dem¬ 
onstrate  compliance  with  the  specification,  the 
necessary  system  interfaces,  and  any  design  con¬ 
straints. 

(2)  Allocated  Configuration  Identif ication( AC  I ) .  Later 
in  the  development  process,  the  requirements  of  the 
FCI  are  allocated  to  individually-identified  sub¬ 
sets  called  configuration  items(CIs)  for  hardware 
and  computer  program  configuration  items (CPC Is)  for 
software.  These  are  normally  identified  during  the 
validation  phase  as  a  part  of  the  system  engineering 
process.  This  description  of  "what"  is  to  be  done 
is  documented  in  individual  development  specifica¬ 
tions  (called  Part  I  Specification). 

(3)  Product  Configuration  Identif  icationOCI) .  Finally, 
the  PCI  describes  in  product  specifications  (called 
Part  II  Specification)  the  "as-built"  configuration 
for  CIs  and  CPCIs.  This  specification,  along  with 
technical  orders  and  drawings,  is  the  basis  from 
which  changes  to  the  system  are  controlled  after 
deployment. 

(Airborne  Systems  Software  Acquisition  Engineering 
Guidebook  for  Configuration  Management,  1978 « 12) 

The  subject  of  this  research  is  the  process  of  develop¬ 
ing  the  allocated  configuration  identification  for  the  system 
level  requirements  that  are  to  be  implemented  via  software. 
This  process,  called  software  requirements  allocation,  is  of 
interest  to  me  for  two  reasons.  First,  1  experienced  many 
of  the  frustrations  and  disappointments  associated  with  soft¬ 
ware  acquisition  as  a  member  of  a  major  command,  control,  and 
communications (C^)  systems  program  office  in  my  last  assign¬ 
ment.  During  that  four  year  period,  I  participated  in  all 
phases  of  the  system  acquisition  cycle  as  a  Computer  Systems 

Analyst  and  a  System  Integration  and  Test  Monitor.  In  those 
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positions,  I  experienced  several  problems  that  probably  could 
be  attributed  to  the  CPC  I  structure  of  the  system.  Secondly, 
my  next  assignment  is  to  a  Headquarters  Air  Porce  Acquisi¬ 
tion  Study  Group.  Hopefully,  the  lessons  learned  in  my  pre¬ 
vious  assignment  and  the  results  of  this  research  will  con¬ 
tribute  to  improvements  in  the  DoO  weapon  systems  acquisi- 

«  tion  procedure. 

Previous  Studies 

'  The  importance  being  placed  on  weapon  system  software 

acquisition  and  management  by  the  DoD  is  reflected  by  the 
number  of  recent  management  and  technical  papers,  guidebooks, 
conferences,  and  study  groups  either  sponsored  by  the  DoD  or 
participated  in  directly  by  DoD  personnel.  As  early  a3  1964, 
high-level  Air  Force  managers  had  already  recognized  that 
computer  technology  was  developing  rapidly  and  that  expanding 
requirements  would  dictate  a  proliferation  of  applications 
(Drezner,  Shulman,  Ware,  Smith,  Davis,  Reinstedt,  and  Turn, 
1976*3).  Since  then,  several  noteworthy  studies  have  iden¬ 
tified  the  need  for  a  good  system  analysis  of  requirements 
and  proper  configuration  management.  For  example,  the  1972 
study,  CC IP-85,  Information  Processing/Data  Automation  Im¬ 
plications  of  Air  Force  Command  and  Control  Requirements  in 
the  1980's,  identified  requirements  analysis/design/exercise 
technology  as  one  of  the  five  most  critical  problems  needing 
further  research  and  development.  Specifically,  the  execu¬ 
tive  summary  concluded  thati 
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"The  nation's  survival  and  prestige  rest 
continuously  on  the  assumption  that  in¬ 
cidents  similar  to  the  Pueblo  and  Liberty 
incidents  would  not  occur  during  grave 
strategic  confrontations.  Information 
processing  techniques  could  and  should 
be  doing  more  in  the  areas  of  C&C  system 
requirements  analysis,  system  design  and 
system  exercising  to  assure  that  this 
will  not  happen"  (Asch,  e^  a^. ,  1975  b»3-4). 

In  another  DoD  sponsored  study,  there  was  a  strong 
feeling  of  urgency  that  an  energetic  program  of  research  be 
undertaken  to  advance  the  software  art.  Those  in  attendance 
at  the  1973  Symposium  on  the  High  Cost  of  Software  were  in 
strong  agreement  that  direct  and  indirect  software  costa  are 
unnecessarily  high  and  are  growing  rapidly.  In  part,  they 
concluded  thati 

a.  "The  key  to  achieving  understandability  appears  to 
be  structure,  i.e.,  the  partitioning  of  a  program 
into  parts  and  the  organization  of  the  parts  into 
levels  of  abstraction  ...," 

b.  application  of  any  of  the  current  methodologies 
(i.e.,  top-down  design,  modular  decomposition, 
formal  specification,  etc.  )  would  be  unwise  unless 
appropriately  supported-both  organizationally  and 
technically"  (Asch,  et  a^. ,  1975  b»3-24). 

In  addition,  an  Ad  Hoc  Group  of  the  Army  Scientific  Ad¬ 
visory  Panel  reported  that  many  software  problems  are  rooted 
in  an  inadequate  initial  system  design  effort.  It  recom¬ 
mended  an  orderly  system  design,  considering  all  reasonable 
alternative  system  architectures  before  a  development  is 
initiated  (Asch,  e^  ai. ,  1975  bi3-60).  Dr,  Barry  Boehm  drew 
a  similar  conclusion  at  the  1974  Aeronautical  Systems  Soft¬ 
ware  Workshop  when  he  presented  a  table  that  compared 
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alternative  software  design  techniques.  He  strongly  em¬ 
phasized  the  necessity  of  more  work  at  the  software  design 
and  structuring  stage  by  presenting  an  analysis  of  errors  on 
a  large,  good  software  project  (Asch,  et  ,  1975  bi3-40). 

In  another  paper,  entitled,  "Specifications!  A  Y  •  Ef¬ 
fective  Software  Development,"  the  authors  stated,  "A  tech¬ 
nology  needs  to  be  developed  that  permits  the  structured  de¬ 
composition  of  the  system  specification  into  a  complete  and 
consistent  set  of  data  processing  subsystem  requirements 
(Belford,  Bond,  Henderson,  and  Sellers,  1976»72). 

Although  these  and  many  other  papers  have  indicated  a 
need  for  further  research  into  system  structuring  and  the 
process  of  software  requirements  decomposition,  an  indepth 
study  of  these  topics  was  not  discovered  in  a  very  extensive, 
but  not  exhaustive,  literature  search.  However,  require¬ 
ments  decomposition  is  addressed  in  a  few  papers  that  were 
presented  at  various  conferences.  For  example,  Kenneth  G. 
Salter  stated,  "Many  software  errors  are  due  to  design  rather 
than  coding.  The  root  of  design  errors  is  the  lack  of  formal 
methodology  for  developing  final  code  from  top-level  system 
requirements."  He  then  proceeded  to  present  a  methodology 
for  system  decomposition  representing  data  processing  re¬ 
quirements  in  terms  of  functions,  control,  functional  flows, 
and  data.  In  conclusion,  the  author  stated,  "The  methodology 
has  been  applied  to  two  sample  problems,  suggesting  that  it 
may  be  adequate"  (Salter,  1976i91-97). 


In  summary,  the  literature  is  replete  with  studies, 
technical  papers,  and  conference  proceedings  that  address 
problems  associated  with  software  engineering  and  software 
acquisition  management.  However,  none  appear  to  have  ad¬ 
dressed  the  process  of  allocating  the  system  requirements 
into  configuration  items  and  computer  program  configuration 
items. 

Statement  of  the  Research  Problem 

Since  weapon  systems  are  not  procured  as  a  single  iden¬ 
tifiable  system  but  rather  as  separate  end  items  of  contrac¬ 
tor  developed,  federal  supply  stock,  and  commercial  "off- 
the-shelf"  items,  a  number  of  configuration  items  are  nor¬ 
mally  identified  for  each  system.  Furthermore,  the  Air  Force 
definition  of  a  configuration  item  is,  "An  aggregate  of  hard¬ 
ware/computer  programs  or  any  of  its  discrete  portions,  which 
satisfies  an  end-use  function,  and  is  designated  by  the  Gov¬ 
ernment  for  configuration  management"  (AFR  65-3,  1974«24). 

Thus,  the  number  and  composition  of  configuration  items 
is  a  critical  design  issue  since  the  Government's  technical 
management  activities  primarily  focus  on  CIs  and  CPC  Is.  For 
example,  the  developer  is  normally  required  to  prepare  and 
deliver  individual  performance  specifications,  test  plans, 
test  reports,  product  specifications  version  description 
documents,  and  users  manuals  to  the  Government  for  each  CPC I, 
In  addition,  each  Cl  and  CPC  I  undergoes  individual  design 

reviews  and  configuration  audits.  Successful  completion  of 
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these  milestones  are  often  misinterpreted  by  the  System  Pro¬ 
gram  Office (SPO)  as  control  of  the  development  effort  and 
visibility  into  the  system  development  process.  In  reality, 
the  Government  has  neither  control  nor  visibility  since 
these  documents,  reviews,  and  audits  offer  very  little  in¬ 
dication  of  the  system's  ultimate  performance  in  an  inte¬ 
grated  environment. 

This  situation  and  the  previously  stated  need  for 
further  research  into  the  process  of  allocating  system  re¬ 
quirements  into  CIs  and  CPC  Is  define  the  problem  area  which 
is  the  subject  of  this  research.  Specifically,  the  thesis 
is  an  investigation  of  the  present  software  requirements 
allocation  process  and  an  evaluation  of  an  alternate  method 
of  selecting  CPCIs. 

Objectives  of  the  Research 

The  overall  objective  of  this  study  is  to  determine  how 
the  data  processing  requirements  of  a  system  are  allocated 
to  CPCIs  and  to  investigate  the  feasibility  of  an  alternate 
methodology.  Specifically,  the  research  will  attempt  toi 

a.  determine  the  DoD  and  A F  policies  and  procedures 
for  CPCI  selection  on  a  major  system  acquisition, 

b.  define  what  criteria  are  currently  used  by  SPOs  and 
contractors  to  allocate  system  requirements  into 
CPCIs. 

c.  determine  the  advantages  end  disadvantages  of  those 
criteria. 

d.  determine  how  the  selected  CPCI  structure  impacts 
the  overall  acquisition  management  process. 
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e.  determine  the  feasibility  and  evaluate  the  potential 
impacts  of  CPC  Is  defined  in  terms  of  system  versions 
or  models,  each  of  which  contains  end-use  system 
functional  capabilities.  This  technique,  called 
"horizontal  allocation,"  would  be  developed  on  an 
incremental  basis  in  which  each  "build"  provides  a 
system  capability  that  is  totally  operational, 
rather  than  pieces  and  parts  that  are  only  partially 
operational. 

Scope  and  Limitations  of  the  Research 

This  study  is  limited  to  an  investigation  of  the  CPC I 
selection  criteria  used  on  large  software- intensive  weapon 
systems.  In  other  words,  information  is  not  included  for 
small  systems  that  contain  limited  or  no  software,  and  for 
large  systems  where  only  one  "natural"  CPC  I  structure 
exists.  Likewise,  the  large  Automatic  Data  Process ing(ADP) 
systems  that  are  acquired  under  the  Air  Force  300-series 
regulations  are  excluded  from  this  research.  To  include 
these  would  expand  the  scope  of  this  research  beyond  the 
limited  time  and  resources  available.  Therefore,  the  persons 
interviewed  are  primarily  limited  to  Electronic  Systems  Di- 
vision(ESD),  Aeronautical  Systems  Division(ASD) ,  Headquarters 
Air  Force  Systems  Command (HQ  AFSC),  Federal  Contract  Re¬ 
search  Centers  (such  as  MITRE),  and  those  contractors  that 
are  currently  developing  (or  have  in  the  recent  past  de¬ 
veloped)  large  software-intensive  systems. 

This  research  will  also  be  limited  in  that  some  sensi¬ 
tive  information  may  not  be  available  to  the  researcher, 
especially  on  existing  programs  that  are  experiencing  prob¬ 
lems.  Similarly,  the  person(s)  familiar  with  the  criteria 
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used  for  selecting  CPC  Is  on  a  program  may  have  departed  the 
program  and  left  no  records. 

Another  limitation  is  the  overall  subjective  nature  of 
the  interview  instrument.  In  other  words,  most  questions 
request  the  respondent's  opinion  and  then  ask  why  he  has 
that  opinion.  Despite  this  constraint,  some  statistical 
analysis  is  possible,  especially  on  the  evaluation  of  the 
horizontal  allocation  methodology. 

Finally,  some  of  the  interviews,  especially  with  con¬ 
tractor  personnel,  were  conducted  by  phone  due  to  the  limited 
time  and  resources  available  for  travel  to  their  different 
locations. 

Assumptions 

A  significant  amount  of  the  data  required  for  this  re¬ 
search  effort  was  collected  by  interviews,  both  personal  and 
telephone.  Therefore,  an  underlying  assumption  is  that  each 
person  interviewed  truthfully  expressed  his  opinion  of  the 
situation  on  his  program. 

Thesis  Organization 

This  thesis  is  organized  into  six  chapters.  This  in¬ 
troductory  section  constitutes  Chapter  I.  The  methodology 
used  to  accomplish  the  research  is  described  in  '".irter  II. 
The  advantages  of,  disadvantages  of,  and  rationale  for  se¬ 
lecting  the  methodology  are  described.  Then,  Chapter  III 
presents  a  review  of  the  present  DoD  and  AF  policies  and 
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I 

procedures  on  the  selection  of  CPCIs  for  a  large  software- 
intensive  weapon  system.  In  addition,  the  directives  which 
govern  the  acquisition  and  management  of  computer  resources 
during  the  development,  acquisition,  deployment,  and  support 
of  major  defense  systems  are  summarized  in  order  to  estab¬ 
lish  the  proper  framework  for  discussing  the  CPCI  selection 
process.  The  relevant  studies,  technical  reports,  proceed¬ 
ings  of  conferences,  textbooks,  and  management  guidebooks 
are  reviewed  in  Chapter  IV.  Chapter  V  presents  and  discusses 
the  interview  results.  Finally,  Chapter  VI  contains  a  sum¬ 
mary  of  and  some  conclusions  on  the  process  of  software  re¬ 
quirements  allocation  on  a  large  software-intensive  system. 

In  addition,  some  recommendations  on  how  the  system  acquisi¬ 
tion  process  in  the  DoD  could  be  improved  are  included. 
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II.  Research  Methodology 


This  thesis  represents  an  investigation  of  one  portion 
of  the  process  of  allocating  system  functional  and  perfor¬ 
mance  requirements  to  individually  identified  subsets  for 
purposes  of  managing  their  development.  These  subsets, 
called  configuration  items (C Is)  for  hardware  and  computer 
program  configuration  items(CFCIs)  for  software,  are  normally 
identified  during  the  early  stages  of  a  system  acquisition. 
The  portion  of  this  allocation  process  that  was  of  concern 
in  this  research  effort  was  the  selection  of  CPCIs  on  a  ma¬ 
jor  weapon  system  that  is  software-intensive.  More  specifi¬ 
cally,  the  research  focuses  on  the  criteria  used  in  making 
the  selection  decision  and  the  feasibility  and  potential 
impacts  of  an  alternate  methodology.  Thus,  the  objective  of 
this  thesis  was  to  answer  the  following  questions i 

a.  What  are  the  DoD  and  AF  policies  and  procedures  for 
CPCI  selection  on  a  major  system  acquisition? 

b.  What  criteria  are  currently  used  by  system  program 
off ices (SPOs )  and  contractors  to  allocate  system 
requirements  into  CPCIs? 

c.  What  are  the  advantages  and  disadvantages  of  those 
criteria? 

d.  How  does  the  selected  CPCI  structure  impact  the 
overall  acquisition  management  process? 

e.  Is  horizontal  allocation  of  requirements  to  be  im¬ 
plemented  via  software  feasible?  Also,  what  are  the 
potential  impacts  of  defining  CPCIs  in  such  a 
manner? 

In  pursuit  of  the  answers  to  these  questions,  a  very 
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extensive  literature  search  and  a  serai-structured  interview 
questionnaire  were  used  for  collecting  the  research  data. 
Chapters  III  and  IV  contain  the  results  of  the  literature 
search,  and  Chapter  V  presents  and  analyzes  the  interview 
results.  The  answers  to  these  questions  are  then  summarized 
in  Chapter  VI  along  with  the  attendant  conclusions  and  rec¬ 
ommendations.  But  first,  the  remainder  of  this  chapter  pre¬ 
sents  a  brief  description  of  the  literature  search  and  a 
detailed  explanation  of  the  methodology  used  to  execute  the 
interviews. 

The  Literature  Search 

The  literature  search  was  performed  to  develop  a  syn¬ 
thesized  description  of  the  software  requirements  allocation 
process  currently  in  use.  Policies,  procedures,  guidelines, 
criteria  for  allocation,  and  the  associated  impacts  were 
typical  categories  of  information  searched  for.  Although 
the  research  topic  was  limited  to  the  selection  of  CPC  Is  on 
a  major  weapon  system  that  is  software-intensive,  over  300 
references  were  identified  and  evaluated  for  applicability. 

Of  those,  215  were  selected  as  relevant  to  this  research 
project.  They  included  Congressional  hearings  1  GAO  reports t 
DoD  and  AF  policies,  regulations,  pamphlets,  and  guidebooks! 
military  standards!  technical  reports!  conference  proceedings! 
studies!  textbooksi  magazine  articles!  and  "lessons  learned." 
The  majority  of  this  published  information  addressed  the  ac¬ 
quisition  and  management  of  computer  resources  during  the 

17 


I 


1 


•4 


development,  acquisition,  deployment,  and  support  of  major 
defense  systems.  Thus,  the  problems,  experiences,  lessons 
learned,  and  alternative  techniques  associated  with  the  CPC  I 
selection  process  were  often  embedded  in  the  discussions  of 
requirements  analysis,  system  design,  software  engineering, 
acquisition  management,  and  software  contracting. 

The  Interview  Questionnaire 

It  is  often  claimed  that  the  use  of  effective  acquisi¬ 
tion  management  techniques  can  significantly  reduce  the  cost 
and  development  time  for  a  major  defense  system.  In  the  past 
20  years,  only  a  few  systems  have  been  delivered  to  the  user 
within  the  original  cost  and  schedule  estimates.  Therefore, 
two  questions  that  often  arise  are i  What  is  the  root  cause 
of  these  failures »  and,  what  effect  do  the  various  manage¬ 
ment  decisions  have  on  the  overall  success  of  the  program? 

To  Investigate  one  of  those  decisions  (i.e.,  the  CPCI  se¬ 
lection  decision),  an  interview  questionnaire  was  designed 
to  capture  both  objective  and  subjective  data  from  Govern¬ 
ment  and  industry  personnel  experienced  in  the  various  dis¬ 
ciplines  of  software  development  and  management.  In  addi¬ 
tion  to  appraising  current  and  recent  programs,  these  "ex¬ 
perts"  were  asked  to  evaluate  the  feasibility  and  potential 
impacts  of  horizontal  allocation  of  software  requirements. 

The  initial  interview  questions  were  based  on  the  re¬ 
searcher's  four  years  of  experience  in  the  development  and 

management  of  computer  resources.  During  the  development  of 
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the  questionnaire,  some  questions  were  added  and  some  were 
deleted  as  additional  knowledge  and  insight  were  obtained 
through  discussions  with  the  research  advisors.  The  result 
was  a  semi-structured  questionnaire  containing  only  35  ques¬ 
tions.  However,  by  using  "packing  techniques"  approximately 
73  responses  were  possible.  Although  this  number  is  not 
overwhelming,  several  of  the  questions  solicited  a  specific 
response  and  then  asked  for  supporting  rationale  in  the  form 
of  a  short  narrative.  The  inclusion  of  these  open-ended 
questions  with  multiple  choice  or  fixed  alternative  types 
define  the  interview  questionnaire  as  "semi-structured."  In 
other  words,  the  interview  questions  were  standardized  so 
that  each  respondent  was  addressing  the  same  topics i  and  each 
respondent  was  provided  an  opportunity  to  present  his  sub¬ 
jective  opinions  on  several  questions.  In  addition,  each 
individual  interviewed  was  promised  anonymity.  To  insure 
this  anonymity,  an  individual  and  his  organization  are  not 
identified  with  his  comments. 

The  questionnaire  was  divided  into  three  sections.  Sec¬ 
tion  I  consisted  of  11  individual  demographic  questions, 
such  as  organizational  level  of  assignment,  type  of  job,  ex¬ 
perience  level,  grade,  educational  level,  number  of  years 
involved  in  system/software  development,  and  number  of  system 
acquisition  programs  associated  with.  Section  II  contained 
two  types  of  questions.  First,  the  respondent  was  asked  to 
identify  the  program  he  is  presently  (or  was  last)  associated 
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with  and  then  to  describe  it  in  terms  of  its  characteristics. 
For  example,  the  questions  covered  total  development  cost, 
number  of  CPC Is,  number  of  lines  of  code,  cost  ratio  of 
software  to  hardware,  and  percentage  of  software  develop- 
ment/c on tract  that  is  complete.  Then,  several  questions  ad¬ 
dressed  the  criteria  used  for  allocating  system  software  re¬ 
quirements  to  CPC  Is  on  the  identified  program.  Finally, 
Section  III  consisted  of  5  questions  that  addressed  the  fea¬ 
sibility,  perceived  effectiveness,  and  implementation  of 
horizontal  allocation  as  an  alternate  technique  for  alloca¬ 
ting  software  requirements  to  CPC  Is.  The  perceived  effec¬ 
tiveness  question  asked  the  respondent  to  evaluate  horizon¬ 
tal  allocation  on  the  basis  of  its  impact  on  12  parameters. 
They  weret  debugging  and  testing  efficiency,  maintenance 
cost,  morale  of  software  experts,  software  integration,  com¬ 
plexity  of  the  decomposition  process,  training  effectiveness, 
cost,  development  time,  performance,  management  visibility, 
software  quality,  and  early  problem  identification.  Indi¬ 
vidual  responses  were  scored  on  a  simple  1  to  4  scale  where 
1  equals  "not  effective,"  2  equals  "somewhat  effective," 

3  equals  "moderately  effective,"  and  4  equals  "very  effec¬ 
tive,  " 

To  eliminate  ambiguity,  a  brief  description  of  the  hor¬ 
izontal  allocation  CPC  I  selection  technique  was  included  in 
Section  III.  The  description  included  a  definition  and 
methodology  for  developing  CPC Is  selected  by  this  alternate 
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technique.  In  addition,  an  example  of  a  simple  Spacetrack 
System  was  attached  to  the  questionnaire  to  explain  the  sa¬ 
lient  characteristics  of  horizontal  allocation.  This  ex¬ 
ample,  along  with  the  questionnaire,  are  included  as  Appen¬ 
dix  D  to  this  thesis. 

Method  of  Conducting  the  Interview 

The  interview  questionnaire  was  designed,  written, 
tested,  and  implemented  in  the  spring  and  summer  of  1979. 

The  selection  of  potential  respondents  could  have  been  ac¬ 
complished  through  either  a  scientific,  random  method  or  a 
subjective  approach.  The  random  method  was  rejected  for  two 
reasons.  First,  the  potential  respondents  are  widely  dis¬ 
persed  and  with  a  time-consuming  questionnaire,  the  number 
of  people  that  could  be  interviewed  had  to  be  limited.  Se¬ 
condly,  there  was  no  means  for  identifying  the  population  of 
"experts"  who  are  presently  (or  were  in  the  recent  past)  re¬ 
sponsible  for  one  of  the  various  disciplines  of  sol  i«<u'e  de¬ 
velopment  and  management.  Therefore,  a  more  subjective  se¬ 
lection  approach  was  chosen.  Individuals  were  chosen  who 
appeared  to  have  special  knowledge  and  experience  involving 
software-intensive  systems.  Actual  selections  were  based  on 
the  literature  search,  the  researcher's  experience,  rec¬ 
ommendations  of  the  thesis  advisor,  and  recommendations  of 
individuals  initially  interviewed. 


Initial  contact  was  made  in  April  and  May  of  1979  to 
determine  if  an  individual  was  interested,  willing,  and  able 


to  participate.  All  of  the  various  disciplines  associated 
with  software  acquisition  and  management  were  included  in 
the  initial  contacts.  After  many  telephone  calls.  87  of  the 
120  people  contacted  (72.5^)  agreed  to  complete  the  ques¬ 
tionnaire.  They  represented  10  Air  Force  organizations,  2 
Federal  Contract  Research  Centers (FCRC ) ,  and  11  Aerospace 
contractors.  The  number  contacted,  number  that  agreed  to 
respond,  and  percent  that  agreed  to  respond  are  described  in 
Table  I  for  each  of  these  categories. 


TABLE  I 


RATE  OF  AGREEMENT 

TO  RESPOND 

Number 

Contacted 

Number 

Agreed  to 
Respond 

Percent 
Agreed  to 
Respond 

Government 

Personnel 

72 

42 

58.3 

FCRC  Personnel 

22 

20 

90.9 

Contractor 

Personnel 

26 

25 

96.2 

Total 


120 


87 


72.5 


In  addition  to  the  questionnaire,  each  potential  re¬ 
spondent  was  promised  a  cover  letter  that  briefly  outlined 
the  purpose  of  this  study,  structure  of  the  questionnaire, 
need  for  cooperation,  and  an  estimate  of  the  time  required 
to  complete  the  questionnaire.  Since  two  types  of  inter¬ 
views  (personal  and  telephone)  were  Initially  planned,  a 
standardized  letter  was  written  and  mailed  to  all  prospec¬ 
tive  respondents.  The  questionnaire  was  attached  to  the 
letters  for  those  scheduled  for  telephone  interviews  (i.e., 
57  of  the  87  agreeing  to  respond).  In  other  words,  the  re¬ 
searcher  attempted  to  prevent  general  discussion  or  collu¬ 
sion  among  those  to  be  personally  interviewed  by  only  pro¬ 
viding  the  cover  letter.  This  approach  was  abandoned  after 
the  initial  interviews  indicated  that  some  forethought  would 
have  resulted  in  more  accurate  and  comprehensive  answers  to 
some  of  the  questions.  Therefore,  a  second  cover  letter  was 
written  and  forwarded,  along  with  the  questionnaire,  to  all 
those  scheduled  for  personal  interviews.  Both  of  these 
letters  are  attached  to  this  thesis  as  Appendix  C. 

Six  of  the  87  people  agreeing  to  respond  requested  ex¬ 
tra  copies  of  the  questionnaire  for  their  subordinates,  as¬ 
sociates,  or  friends.  As  indicated  in  Table  II,  a  total  of 
32  extras  were  distributed  to  government  and  contractor  per¬ 
sonnel.  Therefore,  a  total  of  119  questionnaires  were  dis¬ 
tributed.  Of  those  only  45  were  returned  by  the  cutoff  date 
(1  August  1979).  This  resulted  in  an  overall  response  rate 
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TABLE  II 


of  51.7#  for  those  personally  contacted  by  the  researcher, 
and  only  37.87*  when  the  additional  copies  were  included.  In 
addition,  5  other  people  provided  specific  comments  on  the 
subject  of  the  thesis,  even  though  they  did  not  complete  the 
questionnaire.  A  listing  of  all  respondents  and  their  re¬ 
spective  organization  is  included  as  Appendix  E. 

Although  these  rates  appear  to  be  low,  especially  in 
light  of  the  close  contact  maintained  between  the  researcher 
and  those  personally  contacted,  they  are  approximately  the 
same  as  achieved  on  two  similar  studies.  For  example,  the 
response  rates  for  a  Software  Data  Collection  Study,  per¬ 
formed  by  System  Development  Corporation (SDC )  for  the  Rome 
Air  Development  Center (RADC)  in  1976,  were  only  30%  from 
internal  SDC  sources  and  20%  from  military  agencies 
(Willmorth,  Finfer,  and  Templeton,  1976il).  Similarly,  the 
University  of  California,  Los  Angeles  Graduate  School  of 
Management  distributed  a  Software  Maintenance  and  Enhance¬ 
ment  Questionnaire  to  120  organizations  and  only  received 
responses  for  69  applications  (Lientz,  Swanson,  and  Tompkins 
1976. ii). 

Accuracy  of  the  Interview  Questionnaire 

The  selection  of  a  method  for  gathering  data  is  one  of 
the  most  important  decisions  a  researcher  will  make,  for  the 
quality  of  his  research  depends  not  only  on  the  adequacy  of 
the  research  design  but  also  on  the  quality  of  the  data 


collection  technique  employed.  Of  the  three  basic  methods 
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available  (i.e.,  controlled  observation,  document/record 
review,  and  survey),  the  survey  provided  the  only  practical 
approach  for  the  objectives  of  this  research.  The  four  ways 
of  conducting  a  survey  are  mail  questionnaire,  personal  in¬ 
terview,  panel  interview,  and  telephone  interview.  As  pre¬ 
viously  indicated,  a  combination  of  the  mail  questionnaire, 
telephone  interview  and  personal  interview  techniques  was 
used  by  this  researcher.  In  other  words,  the  sample  popu¬ 
lation  was  identified,  a  questionnaire  was  mailed  to  the 
prospective  respondent,  and  then  a  personal  interview  or 
telephone  discussion  was  used  to  obtain  answers  to  the  survey 
questions.  The  selection  of  this  approach  was  based  on  sev¬ 
eral  criteria,  such  as  factors  to  be  measured  and  relevance, 
reliability,  validity,  and  cost.  They  are  each  briefly  dis¬ 
cussed  in  the  following  paragraphs. 

Factors  to  be  Measured  and  Relevance.  The  type  of  in¬ 
formation  to  be  collected  often  influences  the  decision  on 
which  data  collection  method  to  use.  Since  the  data  to  be 
captured  included  opinions  and  factual  information,  a  survey 
type  method  was  chosen  as  most  appropriate  for  this  research 
effort.  In  other  words,  the  combination  of  telephone  and 
personal  interviews  was  judged  to  produce  the  most  relevant 
data. 

Reliability.  To  be  useful,  a  data  collection  method 
and  the  rules  for  using  the  data  must  produce  information 
that  is  not  only  relevant  but  correct.  Two  crucial  aspects 


of  correctness  are  reliability  and  validity.  A  method  is 
reliable  to  the  extent  that  it  yields  consistent  results  in 
repeated  independent  applications.  This  assumes  that  any 
random  influence  which  tends  to  make  measurements  different 
on  repeated  applications  is  a  source  of  measurement  error 
(Nunnally,  1 9671 206).  Time  is  not  available  for  duplication 
of  efforts  which  would  identify  the  random  influences.  How¬ 
ever,  this  questionnaire  is  believed  to  be  reliable  since  no 
differences  were  detected  between  early  and  late  responses. 

Validity.  It  is  possible  for  measurement  scores  to  be 
highly  reliable  without  being  valid.  That  is,  a  question¬ 
naire  may  be  consistent  without  measuring  what  it  is  sup¬ 
posed  to.  But  the  converse  is  not  true.  Data  cannot  be 
highly  valid  without  also  being  reliable  (Nunnally,  1967174). 
Validity,  then,  refers  to  whether  an  instrument  measures 
what  it  is  designed  to  measure.  Determining  the  validity  of 
some  measurement  instruments  is  rather  easy.  For  example, 
verifying  the  proper  performance  of  a  yardstick  as  a  measure 
of  length  is  simple.  However,  validation  of  most  instru¬ 
ments,  especially  those  used  to  measure  human  feelings  and 
behavior,  is  considerably  more  complex.  Establishing  valid¬ 
ity  for  these  instruments  requires  empirical  investigations. 
Furthermore,  the  nature  of  evidence  required  for  the  valida¬ 
tion  depends  on  the  type  of  validity  being  investigated. 

Researchers  are  most  often  interested  in  four  types  of 
validity!  concurrent,  predictive,  content,  and  construct. 
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An  instrument  that  distinguishes  individuals  who  differ  in 
their  present  status  is  said  to  have  concurrent  validity 
(often  referred  to  as  discriminant  validity).  On  the  other 
hand,  researchers  use  an  instrument  with  predictive  validity 
when  they  wish  to  distinguish  individuals  who  will  differ  in 
the  future.  For  some  other  instruments,  validity  depends 
primarily  on  the  adequacy  with  which  a  specified  domain  of 
content  is  sampled.  Content  validity,  then,  requires  a  com¬ 
plete  specification  of  the  universe  of  possible  responses 
and  all  the  test  items  that  might  be  used  to  measure  it. 
Since  the  number  of  potential  test  items  may  approach  in¬ 
finity,  determining  content  validity  is  frequently  impos¬ 
sible.  Finally,  a  researcher  is  often  interested  in  deter¬ 
mining  a  basis  for  inferring  the  degree  to  which  an  individ¬ 
ual  possesses  some  characteristic  or  trait.  Since  such 
characteristics  usually  cannot  be  identified  with  a  single 
behavior,  a  variety  of  behaviors,  called  "constructs,"  are 
assumed  to  correlate  with  one  another  such  that  the  charac¬ 
teristic  can  be  identified.  The  process  of  validating  this 
type  of  measuring  instrument  is  called  construct  validity 
(Selltiz,  Wrightsman,  and  Cook,  1976tl?l-179). 

The  type  of  validity  this  researcher  primarily  con¬ 
cerned  himself  with  was  content  validity.  Although  empir¬ 
ical  tests  to  demonstrate  the  validity  of  the  interview 
questionnaire  were  not  practical,  it  is  assumed  to  provide 
valid  data  for  several  reasons.  First,  the  questionnaire 
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was  reviewed,  critiqued,  and  pretested  by  three  software 
engineers  assigned  to  the  Directorate  of  Engineering  at  the 
Aeronautical  Systems  Division,  Wright-Patterson  Air  Force 
Base,  Ohio.  Although  these  engineers  are  primarily  con¬ 
cerned  with  the  acquisition  and  management  of  computer  re¬ 
sources  embedded  in  aircraft  systems,  they  are  familiar  with 
the  techniques  and  issues  associated  with  software  develop¬ 
ment  for  various  applications.  As  a  result  of  their  efforts, 
some  questions  were  clarified  and  others  were  added. 

Secondly,  advance  letters,  refusal  rates,  sample  se¬ 
lection,  interviewer  bias,  and  interviewee  cooperation  have 
all  been  demonstrated  to  have  an  affect  on  the  validity  of 
data  obtained  via  personal  and  telephone  interviews  (Cooper, 
1978 i 14-17).  To  reduce  the  adverse  impacts,  several  pre¬ 
cautions  were  taken  during  the  data  collection  phase.  For 
example,  an  individual  was  only  included  in  the  sample  if  he 
expressed  an  interest  in  the  topic  of  the  thesis.  Then,  ex¬ 
planatory  cover  letters  (a  type  of  advance  letter)  were 
mailed  to  each  prospective  respondent.  These  precautions  in 
conjunction  with  the  outstanding  cooperation  of  those  being 
interviewed  and  the  researcher's  efforts  to  remain  unbiased 
support  the  validity  assumption. 

Cost.  The  cost  associated  with  conducting  face-to-face 
interviews  often  forbids  administering  personal  interviews  to 
a  large  sample  population.  In  other  words,  the  time  and 
travel  cost  associated  with  questioning  each  individual 


separately  is  often  prohibitive.  Since  telephone  interview¬ 
ing  is  generally  considered  to  be  significantly  cheaper  and 
to  exhibit  a  high  inter-interviewer  reliability,  a  combina¬ 
tion  of  the  two  interview  methods  was  chosen  for  this  re¬ 
search.  This  method  appears  to  have  resulted  in  data  that 
is  relevant,  reliable,  and  valid. 

Problems  with  the  Interview  Questionnaire  and  Methodology 
Although  much  effort  was  expended  trying  to  ensure  the 
best  designed,  written,  and  implemented  questionnaire,  some 
problems  were  experienced.  For  example,  only  a  small  sample 
of  the  people  and  organizations  developing  and  managing  soft¬ 
ware  -intensive  systems  was  interviewed.  Thus,  one  cannot 
conclusively  state  that  the  conclusions  drawn  in  this  thesis 
are  true  in  all  cases.  Similarly,  several  questionnaires 
were  returned  with  some  questions  unanswered.  Several  rea¬ 
sons  can  be  hypothesized  for  this.  They  include i  the  level 
of  detail  was  unknown  to  the  respondent,  the  data  was  con¬ 
sidered  proprietary,  the  respondent’s  fear  of  reprisal  (al¬ 
though  anonymity  was  promised),  and  the  respondent  did  not 
understand  the  question.  This  author  concluded  that  the 
level  of  detail  was  unknown  to  the  interviewee  since  "N/A" 
or  "unknown"  was  written  beside  many  questions.  Further¬ 
more,  a  sufficient  number  of  responses  was  received  to  allow 
statistical  analysis,  even  though  the  rate  of  blank  re¬ 
sponses  was  as  high  as  37£  for  a  couple  of  highly  technical 
questions. 
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The  last  problem  experienced  with  the  questionnaire 
concerned  the  definitions  of  "requirements  allocation"  and 
"requirements  decomposition."  Webster’s  New  Collegiate 
Dictionary  defines  allocation  as,  "To  apportion  for  a  spe¬ 
cific  purpose  or  to  particular  persons  or  things »  DISTRI¬ 
BUTE  tasks  among  human  and  automated  components"!  and  de¬ 
composition  as,  "To  separate  into  constituent  parts  or  ele¬ 
ments  or  into  simpler  compound"  (Webster’s  New  Collegiate 
Dictionary.  1976130,  29*0.  When  applied  to  the  requirements 
of  a  system  specification  and  the  act  of  defining  components 
of  the  system,  these  terms  have  clear  meaning.  Unfortunately, 
this  clear  distinction  is  not  projected  in  the  literature, 
both  regulatory  and  non-regulatory.  As  a  result,  a  few  of 
the  respondents  requested  clarification  before  completing 
the  questionnaire.  In  each  case,  allocation  was  described 
as  the  act  of  apportioning  the  system  performance  and  func¬ 
tional  requirements  to  individually-identified  subsets  for 
purposes  of  managing  their  development.  The  identification 
of  configuration  items  was  cited  as  the  typical  result  of 
this  apportioning  process.  On  the  other  hand,  decomposition 
was  described  as  the  hierarchial  separation  of  system  or 
subsystem  requirements  into  units  of  development,  such  as 
functions,  tasks,  modules,  subroutines,  components  and  in¬ 
structions.  In  each  case,  these  explanations  seemed  to 
clarify  the  intent  of  the  questionnaire.  Therefore,  the 
assumption  of  validity  in  the  data  appears  to  be  a  reasonable 


one 
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Data  analysis  is  an  integral  part  of  any  research  pro* 
ject.  If  survey  data  has  been  collected,  the  selection  of 


an  analysis  technique  is  likewise  very  important.  Since 
this  thesis  is  highly  management  oriented,  the  primary  ob¬ 
jective  in  analyzing  the  data  gathered  in  this  research  is 
to  determine  if  an  individual's  demographics  or  the  charac¬ 
teristics  of  the  systems  program  he  evaluated  has  any  affect 
on  his  opinion  of  the  utility  of  horizontal  allocation.  A 
variety  of  techniques  are  available  for  performing  analysis 
on  the  data  collected.  Each  technique  chosen  and  the  neces¬ 
sary  preliminary  steps  are  identified  in  Figure  2,  Further¬ 
more,  they  are  all  described  in  the  following  paragraphs, 
with  one  exception.  In  other  words,  the  data  collection  step 
is  not  described  in  this  section  since  it  was  discussed  in 
the  previous  section  entitled,  Method  of  Conducting  the  In¬ 
terview. 

Type  of  Data.  The  type  of  data  gathered  in  this  i->- 
search  is,  of  necessity,  both  subjective  and  objective. 
Therefore,  the  questions  must  be  separated  into  those  re¬ 
quiring  statistical  analysis  and  those  to  be  subjectively 
analyzed.  Whenever  possible,  statistical  methods  were  used 
since  they  provide  assistance  in  understanding  complex  real- 
world  processes  and  provide  a  standardized  "language"  for 
exchanging  information  about  those  processes  among  research¬ 
ers  and  decision  makers  (McNichols,  1978 il-l).  The  respon¬ 
dent's  answers  to  the  questions  that  only  lend  themselves  to 
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Figure  2.  Data  Analysis  Plan 
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Figure  2  (continued).  Data  Analysis  Plan 
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qualitative  analysis  are  categorized,  analyzed,  and  summa¬ 
rized  in  an  attempt  to  reach  some  logical  conclusions  and, 
in  some  cases,  recommendations.  In  addition,  the  summaries 
are  supported  with  comments,  experiences,  and  lessons 
learned.  As  previously  indicated,  the  only  identification 
attached  to  an  individual's  response  is  that  it  represents 
the  opinion  of  a  Government,  FCRC,  or  Contractor  expert. 

Data  Preparation.  Upon  receipt  of  each  interview  ques¬ 
tionnaire,  the  answers  to  those  questions  requiring  statis¬ 
tical  analysis  were  coded  and  punched  onto  a  standard  IBM 
card.  The  data  from  a  single  respondent's  questionnaire  was 
punched  onto  a  single  card  and  is  referred  to  as  a  case 
throughout  the  remainder  of  this  thesis.  Then,  each  case 
was  edited  for  keypunch  errors  before  being  analyzed  by  the 
Statistical  Package  for  the  Social  Sciences (SPSS)  procedures. 
This  integrated  system  of  computer  programs  is  designed  to 
provide  the  researcher  with  a  comprehensive  set  of  automated 
statistical  techniques  commonly  used  in  analyzing  social 
science  data  (Nie,  Hull,  Jenkins,  Steinbrenner,  and  Bent, 

1978 il).  The  SPSS  programs  resident  on  the  Aeronautical 
Systems  Division  CYBER  175  computer  system  at  Wright- 
Patterson  Air  Force  Base  were  used  by  this  researcher  for 
analyzing  the  interview  data. 

Descriptive  Statistics.  The  basic  distributional  char¬ 
acteristics  of  each  variable  to  be  used  in  the  subsequent 
statistical  analysis  were  computed  with  the  SPSS  Frequency 


Procedure.  More  specifically,  information  on  the  distriou- 
tion,  variability  and  central  tendencies  of  each  variable 
was  obtained  through  selection  of  the  "All"  statistics  and 
histogram  options.  In  addition  to  the  frequencies,  this  re¬ 
searcher  is  specifically  concerned  with  the  following  uni¬ 
variate  descriptive  statistics i  means,  modes,  and  standard 
deviations. 

Analysis  of  Variable  Interdependency,  After  the  de¬ 
scriptive  statistics  were  studied,  two  analysis  objectives 
were  chosen  as  applicable  to  the  data.  The  first  one,  anal¬ 
ysis  of  variable  interdependency,  provides  a  measure  of  the 
strength  of  relationship  between  two  variables.  Two  tech¬ 
niques  were  used  for  determining  this  relationship,  depending 
on  the  type  of  scale  used  in  measuring  the  data.  They  are 
the  Pearson  Product-Moment  Correlation  Coefficient  for  pairs 
of  ordinal-scaled  variables!  and  the  Chi-square  test  for  nom¬ 
inal-scaled  variables  (often  called  categorical  data).  These 
techniques  were  performed  using  the  SPSS  procedures  PEARSON 
CORR  and  CROSSTABS,  respectively. 

Analysis  of  Statistical  Significance.  The  second  anal¬ 
ysis  objective  was  to  determine  the  statistical  significance 
of  different  responses  to  some  of  the  interview  questions. 

In  other  words,  the  objective  was  to  determine  if  signifi¬ 
cantly  more  than  half  of  the  sample  population  agree  that 
horizontal  allocation  is  feasible  and  should  be  implemented 
on  a  future  system.  To  make  this  determination,  a  one-tailed 


36 


t  test  was  used.  There  are  two  assumptions  associated  with 
that  test.  Both  of  them  (random  sample  and  normal  distri¬ 
bution)  are  assumed  to  be  satisfied. 


Chapter  Summary 

This  chapter  presented  a  description  of  the  research 
methodology  used  to  investigate  the  process  of  allocating 
software  requirements  to  CPC  Is  on  a  software -intensive 
system.  In  summary,  an  extensive  literature  search  was  used 
to  ascertain  the  DoD  and  A F  policies  and  procedures  on  CPCI 
selection  on  a  major  system  acquisition.  Then,  a  semi- 
structured  interview  questionnaire  was  designed,  written, 
and  implemented  to  investigate  the  CPCI  selection  process 
currently  used  by  SPOs  and  contractors.  In  addition,  the  . 
questionnaire  was  used  to  capture  the  perceived  effectiveness 
of  an  alternate  approach  to  CPCI  selection,  called  horizontal 
allocation. 

A  description  of  the  format  of  the  questionnaire,  method 
of  conducting  the  interview,  and  accuracy  of  the  question¬ 
naire,  is  also  included  in  this  chapter.  The  questionnaire 
was  divided  into  three  sections!  demographics,  program 
characteristics  and  criteria  currently  used  for  allocating 
software  requirements  to  CPCIs,  and  the  feasibility  and  po¬ 
tential  impacts  of  horizontal  allocation.  The  questionnaire 
was  reviewed  and  critiqued  by  three  software  engineers,  and 
it  was  assumed  to  be  basically  reliable  and  valid.  Since 
the  questionnaire  was  designed  with  its  accuracy  in  mind  and 
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no  major  problems  were  encountered  in  conducting  the  tele¬ 
phone  and  personal  interviews,  this  assumption  appears  to  be 
reasonable. 

In  addition,  a  comprehensive  data  analysis  plan  was 
outlined  in  Figure  2  anu  described  in  this  chapter.  In  sum¬ 
mary,  two  types  of  analysis  (qualitative  and  quantitative) 
were  used.  The  qualitative  analysis  consisted  of  catego¬ 
rising,  analysing,  and  summarising  the  answers  expressed  by 
the  respondents.  On  the  other  hand,  three  statistical  anal¬ 
ysis  techniques  were  applied  to  the  quantitative  data.  They 
were  Pearson  Product-Moment  Correlation,  Chi-square  test, 
and  Students  t  test.  Additionally,  univariate  descriptive 
statistics  for  each  of  the  variables  were  obtained  via  the 
SPSS  Frequency  Procedure  and  the  appropriate  options. 


III.  DoD  and  Air  Force  Policies  and  Procedures 


The  acquisition  and  management  of  computer  resources 
within  the  Federal  Government  are  controlled  by  a  complex 
hierarchy  of  policies  and  directives.  At  the  top  of  that 
hierarchy  is  Public  Law  98-306  which  provides  the  basic 
structure  and  concepts  for  the  government-wide  system  of 
Automated  Data  Processing  Equipment (ADPE)  management.  In 
the  Executive  branch,  this  law  has  been  implemented  through 
the  issuance  of  special  rules  by  the  Office  of  Management 
and  Budget ( OMB ) ,  the  General  Services  Administration(GSA) , 
and  the  individual  federal  agencies.  To  implement  these 
rules,  the  DoD  has  issued  a  series  of  directives,  instruc¬ 
tions,  and  manuals,  and  the  AF  has  published  a  series  of 
regulations,  pamphlets,  and  guidelines  dealing  with  computer 
resources.  In  addition  to  these  specialized  rules,  the  ac¬ 
quisition  and  management  of  computer  resources  are  also  sub¬ 
ject  to  the  general  rules  covering  all  federal  procurement 
and  property  management. 

This  chapter  presents  an  overview  of  the  regulatory 
structure  which  governs  the  acquisition  and  management  of 
computer  resources  during  the  development,  acquisition,  de¬ 
ployment,  and  support  of  major  defense  systems.  More  spe¬ 
cifically,  the  DoD  and  AF  policies  and  procedures  on  the  se¬ 
lection  of  CPC Is  for  a  large  software -intensive  weapon  system 
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is  the  primary  concern  of  this  chapter.  For  example,  a  com¬ 
mand,  control,  and  communications (C^)  system  is  a  large 
software-intensive  weapon  system,  and  it  can  inherently  be 
partitioned  into  more  than  one  set  of  CPCIs,  Before  dis¬ 
cussing  the  DoD  and  AF  policies  and  directives  governing 
this  process,  a  brief  summary  of  how  they  evolved  and  how 
they  interface  with  other  governmental  directives  is  re¬ 
quired  for  proper  orientation.  Figure  3  provides  an  over¬ 
view  of  these  directives  and  how  they  interact  (Kuo,  Durieux, 
Forsythe,  Leary,  Marciniak,  Putman,  and  Spiro,  1977»Atch  l). 

Historical  Perspective!  ADPE  Versus  ECS 

In  the  1960's,  the  primary  concern  for  computer  re¬ 
source  management  was  the  proliferation  of  computers.  At 
that  time,  computers  primarily  automated  functions  that  had 
been  accomplished  manually  (e.g. ,  payroll,  supply).  They 
were  acquired  to  accomplish  specific  tasks  and  were  viewed 
as  special  purpose  machines.  But  there  was  a  great  similar¬ 
ity  among  types  of  user  functions,  resulting  in  the  applica¬ 
tion  of  computers  across  functional  lines.  This  was  the  be¬ 
ginning  of  the  "General  Purpose"  computer.  The  resulting 
growth  of  automated  applications  was  undisciplined  and  some¬ 
times  chaotic.  During  this  period,  under  the  auspices  of 
legislation  called  the  "Brooks  Bill"  (later  pa^nd  as  P.L. 
89-306),  and  subsequent  DoD  Directives,  authority  was  dele¬ 
gated  to  the  Assistant  Secretary  of  Defense (Comptroller ) 


and  the  Secretary  of  the  Air  Force  for  Financial  Management 

40 


PI  B9-306 


(SAP/PM)  to  control  the  purchase  of  the  off-the-shelf  com¬ 
mercially  available  computers.  The  objective  of  this  ap¬ 
proach  was  to  prevent  costly  duplication  and  waste  of  auto¬ 
matic  data  processing(ADP)  resources  (Kuo,  gl. ,  19??«C-l). 

At  the  same  time,  technology  progressed  to  the  point 
where  small  computers  were  installed  as  embedded  components 
in  most  major  defense  systems.  This  special  subset  of  com¬ 
puter  applications  are  called  Embedded  Computer  Systems (ECS) 
to  distinguish  them  from  ADP  systems.  They  are  physically 
incorporated  into  a  larger  system  whose  primary  function  is 
not  data  processing  and  their  outputs  generally  include  con¬ 
trol  signals,  as  well  as  computer  data.  In  other  words, 
they  are  integral  to  a  weapon  system  from  a  design,  procure¬ 
ment,  and  operations  viewpoint.  So,  embedded  computers  range 
from  small  units  in  airborne  systems  to  large  units  in 
ground  and  ship  based  command  and  control  systems  (DeRoze, 
1977 »1-1).  Therefore,  excluded  from  embedded  systems  are 
general  purpose,  commercially  available  ADP  resources  as  de¬ 
fined  and  administered  under  OMB  Circular  A-71,  DoD  Direc¬ 
tives  5100.40,  4105.55  and  4i6o.19,  and  the  AP  300-series 
regulations  (Ref.  Figure  3). 

This  led  to  the  evolution  of  a  separate  set  of  policies, 
directives,  and  regulations,  such  as  OMB  Circular  A-109,  DoD 
Directives  5000.1  and  5000.29,  and  AF  Regulation  800-14  (Ref. 
Figure  3).  The  purpose  of  this  series  is  to  establish  guide¬ 
lines  for  the  acquisition  and  support  of  computer  equipment 
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and  computer  programs  employed  as  dedicated  elements,  sub¬ 
systems,  or  components  of  major  systems  developed  or  ac¬ 
quired  under  the  program  management  concept.  Since  the 
governmental  ADP  and  ECS  regulatory  system  is  extraordinarily 
complex,  it  is  beyond  the  scope  of  this  chapter  to  present 
a  complete  and  definitive  review  of  all  the  policies,  reg¬ 
ulations,  and  procedures.  Rather,  a  summarized  and  simpli¬ 
fied  overview  of  the  more  pertinent  ones  is  provided.  Par¬ 
ticular  emphasis  is  placed  on  those  regulations  that  are 
concerned  with  the  allocation  of  system  requirements  to  CPC  Is 
and  how  computer  resources  are  managed  during  the  acquisi¬ 
tion  of  a  major  defense  system.  In  other  words,  general- 
purpose  commercially  available  ADPE  resources  are  excluded. 

In  addition,  significant  conflicts  in  the  literature  are 
highlighted. 

Congress i  Public  Laws  Affecting  ADPE  and  ECS 

Public  Law  89-306.  Numerous  laws  enacted  by  Congress 
have  an  impact  on  the  acquisition  and  management  of  ADPE 
and  ECS.  The  most  important  of  these  is  P.L.  89-306,  In 
1965*  Congress  passed  P.L.  89-306  to  correct  the  pervasive 
mismanagement  of  ADPE  in  the  federal  government.  Between 
1958  and  1^65,  the  Comptroller  General  submitted  more  than 
100  audit  reports  to  Congress  showing  a  pattern  of  ADPE  mis¬ 
management.  Government  agencies  acquired  computers  inde¬ 
pendently  without  regard  to  government-wide  needs.  As  a  re¬ 
sult,  the  government  did  not  receive  any  volume  price 
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discounts  on  its  purchases  although  it  was  the  largest  com¬ 
puter  user  in  the  world.  There  was  little  computer  sharing 
between  agencies  and  many  computers  were  under  utilized. 
Despite  these  and  other  widespread  problems,  little  action 
was  taken  by  the  Executive  branch  beyond  the  issuance  of 
advisory  guidelines.  In  Congress,  the  House  Government 
Activities  Subcommittee,  under  Chairman  Jack  Brooks,  held 
hearings  on  Federal  ADPE  management  in  1963  and  1965.  The 
result  was  the  passage  of  Public  Law  89-306,  commonly  known 
as  the  Brooks  Act,  in  October  1965  (Self,  1978«55-56). 

The  stated  purpose  of  P.L.  89-306  is  "...  to  provide 
for  the  economic  and  efficient  purchase,  lease,  maintenance, 
operation,  and  utilization  of  automatic  data  processing 
equipment  by  federal  departments  and  agencies.  The  intent 
of  Congress  was  to  achieve  a  businesslike  Government-wide 
coordinated  management  effort  ..."  by  providing  a  "delinea¬ 
tion  of  responsibilities  and  stronger  orr  * . national  plan 
for  Government  ADP  management  ..."  (Senate  Report  No,  93^. 
19650877).  Thus,  P.L.  89-306  did  not  establish  specific 
procurement  or  administrative  policy.  Rather,  it  estab¬ 
lished  a  centralized  ADP  management  structure  in  the  Exec¬ 
utive  branch  and  assigned  responsibilities  for  providing 
ADP  management. 

Three  agencies,  OMB,  GSA,  and  the  National  Bureau  of 
Standards (NBS) ,  were  assigned  specific  responsibilities. 

OMB  was  assigned  responsibility  for  policy  and  fiscal 
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matters.  GSA  was  assigned  operational  responsibility  for 
ADPE  procurement,  maintenance,  and  utilization  decisions. 

NBS  was  assigned  the  technical  aspects  of  ADP  management. 
Finally,  certain  responsibilities  were  left  with  the  agencies 
which  utilize  ADPE,  hereafter  referred  to  as  the  user  agen¬ 
cies  (P.L.  89-306|  Senate  Report  No.  938,  196513861,  3877- 
3885). 

An  exception  to  this  centralized  procurement  is  allowed 
by  paragraph  (b)(2)  of  the  law.  It  permits  GSA  to  delegate 
the  lease,  purchase,  or  maintenance  of  individual  ADP  sys¬ 
tems  to  the  user  agency  when  necessary  for  reasons  of  econ¬ 
omy,  efficiency,  national  security,  or  national  defense.  In 
the  implementation  of  P.L.  89-306,  those  computer  resources 
integral  to  weapon  systems  and  specially  designed  computer 
equipment  were  excluded  from  control.  This  specific  exclu¬ 
sion  was  contained  in  OMB  Circular  74-5  and  DoDD  5100.40 
(Ref.  Figure  3).  The  purpose  of  this  exclusion  was  to  retain 
the  authority  of  the  Systems  Program  Manager  to  accomplish 
his  assigned  tasks.  The  impact  of  this  exclusion  and  the 
resulting  regulations  are  summarized  in  later  paragraphs. 

Armed  Services  Procurement  Act  of  1947.  Other  laws  are 
more  general  and  apply  to  all  types  of  property,  including 
computer  resources.  For  example,  the  Armed  Services  Pro¬ 
curement  Act  of  1947  authorized  the  DoD  to  develop  the  Armed 
Services  Procurement  Regulations (ASPR).  While  the  direc¬ 
tives  implementing  P.L.  89-306  are  essentially  concerned 
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with  internal  government  policies,  the  ASPR  deals  primarily 
with  the  procurement  and  contracting  relationship  between 
the  government  and  private  enterprise.  However,  one  source 
of  considerable  confusion  and  conflict  in  the  acquisition 
and  management  of  computer  resources  is  the  ASPR  treatment 
of  computer  software.  In  summary,  software  is  considered 
as  an  item  of  data  in  the  same  manner  as  reports,  forms, 
manuals,  and  specifications  are.  More  specifically,  data 
management  practices  in  DoO  do  not  restrict  data  to  items 
written  or  printed  on  paper i  they  include  information  re¬ 
corded  in  suitable  form  on  any  suitable  medium,  such  as 
film,  photographic  paper,  magnetic  paper  or  tape,  and  in 
digital  or  analog  form  (Searle,  1977 till). 

This  concept  was  reinforced  in  November  1974  when  a  re¬ 
vision  of  the  ASPR  appeared  providing  extensive  new  and  re¬ 
vised  coverage  in  Section  IX  on  rights  associated  with  com¬ 
puter  software.  The  Defense  Procurement  Circular (DPC )  74-3 
states  that  computer  software  falls  within  the  definition  of 
data,  but  is  specifically  excluded  from  the  definition  of 
technical  data.  As  an  item  of  data,  ASPR  requires  the  list¬ 
ing  of  computer  software  on  the  Contract  Data  Requirements 
List,  DD  Form  1423,  as  a  measure  to  protect  the  Government's 
right  in  data.  Although  the  ASPR  appears  to  have  a  basis  in 
legal  decisions  which  have  been  reached  in  determining 
whether  computer  programs  are  things  to  be  copyrighted  and/ 
or  patented,  the  Air  Force  Systems  Command(AFSC)  has 
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Initiated  an  attempt  to  have  the  ASPR  committee  reverse  its 
policies  (Searle,  1977illl-112).  This  decision  was  based 
on  AFSC's  policy  that  computer  software  acquisition  be  sub¬ 
jected  to  the  same  contractual  and  cost  controls  as  hard¬ 
ware.  In  other  words,  software  delivery,  like  hardware, 
should  be  listed  in  the  contract  schedule  with  desired  per¬ 
formance  characteristics  spelled  out  in  the  Statement  Of 
Work(SOW).  Furthermore,  software  is  delivered  as  an  active 
system  component  and  functionally  performs  as  part  of  a 
system.  In  this  respect,  software  is  the  same  as  hardware j 
whereas,  the  documentation  for  hardware  and  software  is 
rightfully  data  since  it  is  supportive  to  the  functioning  of 
the  system  (AFSCRP  800-1,  1975*3). 

Office  of  Management  and  Budget 

The  Office  of  Management  and  Budget (0MB)  has  broad 
policy  and  fiscal  authority  within  the  Executive  branch. 

This  authority  derives  from  two  sources.  First,  as  a  staff 
office  of  the  President,  0MB  inherently  possesses  extensive 
authority  to  implement  Presidential  policy.  Secondly,  0MB 
has  been  assigned  specific  duties  and  responsibilities  under 
certain  laws,  like  P.L.  89-306.  This  policy  and  fiscal  au¬ 
thority  is  traditionally  implemented  through  the  issuance  of 
0MB  circulars.  Several  circulars  which  directly  address  the 
procurement  and  management  of  ADPE  resources  have  been 
issued. 

However,  only  one,  0MB  Circular  A-109,  directly  imp*'  ts 
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the  acquisition  and  management  of  embedded  computer  re¬ 
sources.  This  circular  establishes  policies  to  be  followed 
by  the  Executive  branch  agencies  in  the  acquisition  of  major 
systems.  More  specifically  the  circular  covers  all  the  ac¬ 
quisition  management  activities  from  analysis  of  agency  mis¬ 
sions  to  the  successful  introduction  of  the  system  into  op¬ 
erational  use.  In  addition,  major  system  acquisition  pro¬ 
grams  are  defined  as  those  thati 

a.  are  directed  at  and  critical  to  fulfilling  an  agency 
mission, 

b.  entail  the  allocation  of  relatively  large  resources, 
and 

c.  warrant  special  management  attention. 

The  definition  of  additional  criteria  (including  establish¬ 
ing  relative  dollar  thresholds)  for  the  determination  of 
major  systems  is  at  the  discretion  of  the  Executive  agency 
head  (Clore,  Friedman,  and  Goheen,  1978«42). 

DoD  Directives 

Annual  expenditures  by  the  DoD  on  the  design,  develop¬ 
ment,  acquisition,  management,  and  operational  support  of 
computer  resources  embedded  within  and  integral  to  weapons, 
communications,  command  and  control,  and  intelligence  sys¬ 
tems  are  measured  in  billions  of  dollars  (Asch,  gjt  al. . 

1975  b 13-16).  At  the  same  time,  such  computer  resources 
have  often  presented  critical  cost  and  schedule  problems 
during  the  development  and  acquisition  of  new  defense  sys¬ 
tems.  Even  after  system  deployment,  the  software  has  often 
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proven  unreliable.  To  correct  these  problems  and  to  improve 
the  management  of  embedded  computer  resources  in  ge’^ral, 
new  policies  and  procedures  have  been  implemented  via  DoD 
Directives.  A  brief  summary  of  the  directives  applicable 
to  the  acquisition  and  management  of  computer  resources  in 
a  major  defense  system  follows.  These  summaries  are  pro¬ 
vided  to  establish  the  framework  for  describing  the  Air 
Force  policies  and  procedures  on  the  allocation  of  system 
requirements  to  CPC  Is  in  the  acquisition  of  a  large  software 
intensive  system. 

DoDD  5000.1.  The  Secretary  of  Defense (SECDEF)  issued 
DoDD  5000.1,  "Major  System  Acquisitions,"  to  implement  the 
policies  of  0MB  Circular  A-109.  Specifically,  this  direc¬ 
tive  establishes  policy  for  the  management  of  programs  des¬ 
ignated  as  major  system  acquisitions.  In  addition  to  the 
criteria  of  "criticality"  and  "special  management  attention, 
relative  dollar  thresholds  are  specified  for  the  designation 
of  a  major  system  acquisition.  That  is,  system  programs  in¬ 
volving  anticipated  cost  of  $75  million  in  research,  develop 
ment,  test  and  evaluation (RDTAE)  or  $300  million  in  produc¬ 
tion  are  designated  as  major  system  acquisitions.  The  di¬ 
rective  also  provides  a  guide  for  management  of  less-than- 
major  system  acquisition  programs.  Basically,  the  policy 
outlines  the  system  acquisition  process  including  responsi¬ 
bilities  of  the  SECDEF,  DoD  components,  the  OSD  staff,  and 
the  Defense  Acquisition  Executive.  Key  decision  points  and 


other  major  aspects  of  acquisition  management  are  defined 
(DoDD  5000.1.  1977«l-3). 

DoDD  5000. 2.  DoD  Directive  5000.2,  "Major  System  Ac¬ 
quisition  Process,"  defines  the  policies  and  procedures  for 
DoD  activities  in  support  of  the  Secretary  of  Defense's 
decision-making  process  on  major  system  acquisitions.  This 
directive  supplements  DoD  5000.1  and  establishes  the  Defense 
Systems  Acquisition  Review  Council(DSARC )  charter.  It 
further  defines  the  responsibilities  of  DoD  component  heads 
in  the  identification  and  analysis  of  mission  needs,  and 
outlines  scheduled  program  reviews.  In  addition,  major 
program  documentation,  such  as  the  Mission  Element  Need 
Statement (MENS)  and  the  Decision  Coordinating  Paper(DCP) 
are  described.  Therefore,  DoDD  5000.2  is  a  key  directive 
for  the  acquisition  of  embedded  computer  systems  (DoDD 
5000.2,  1 977 « 1-5). 

DoDD  5000.29.  The  ever  expanding  applications  and  as¬ 
sociated  problems  of  embedded  computer  systems  have  become 
a  concern  at  the  highest  levels  of  the  DoD.  As  a  result, 
new  policies  for  the  management  of  embedded  computer  re¬ 
sources  have  been  developed.  The  Air  Force  has  led  the  ef¬ 
fort  to  establish  controls  and  cognizance  over  the  ECS  area 
by  publishing  AFR  800-14,  "Management  of  Computer  Resources 
in  Systems,"  which  was  followed  by  DoDD  5000.29,  "Management 
of  Computer  Resources  in  Major  Defense  Systems."  These  doc¬ 
uments  recognised  the  need  to  have  different  but  complementary 
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instructions  for  the  overlapping  areas  of  embedded  systems 
and  general-purpose,  commercially  available  automated  data 
processing  systems  (Kuo,  ei  ai. ,  1977  «C-2). 

DoDD  5000,29  was  therefore  written  to  establish  policy 
for  the  management  and  control  of  computer  resources  that 
are  part  of  a  major  system  acquisition  as  defined  in  DoDD 
5000,1,  These  principles  may  be  applied  to  other  systems, 
with  the  exception  of  general-purpose,  commercially  avail¬ 
able  automated  data  processing  systems  which  are  administered 
under  DoD  Directives  4105.55  and  5100.40.  Therefore,  com¬ 
puter  resources  are  defined  for  the  purposes  of  this  paper, 
as  components  which  must  be  managed  as  elements  or  subsystems 
of  major  importance  during  conceptual,  validation,  full- 
scale  development,  production,  deployment,  and  support 
phases  of  the  life  cycle.  Particular  emphasis  on  computer 
software  and  its  integration  with  the  surrounding  hardware 
is  also  required  (DoDD  5000.29,  1 976 * 2 ) . 

Thi3  directive  also  establishes  a  Management  Steering 
Committee  for  embedded  computer  resources  at  the  Office  of 
Secretary  of  Defense (OSD)  level  to  oversee  the  implementa¬ 
tion  of  the  directive  and  the  incorporation  of  its  policies 
into  the  defense  system  acquisition  process.  In  addition, 
policy  is  defined  for  the  following  major  areas  1 

a.  Requirements  validation  and  risk  analysis. 

b.  Configuration  management  of  computer  resources. 

c.  Computer  resources  life  cycle  planning. 
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d.  Support  software  deliverables. 

e.  Milestone  definition  and  attainment  criteria. 

f.  Software  language  standardization  and  control. 

(DoDD  5000.29,  1976*2-4) 

In  addition  to  the  emphasis  placed  on  computer  resources, 
the  configuration  management  policy  established  in  this  di¬ 
rective  is  of  primary  importance  to  the  research  effort. 
Therefore,  the  latter  policy  is  quoted  as  follows* 

Configuration  Management  of  Computer  Re¬ 
sources.  Defense  system  computer  re¬ 
sources,  including  both  computer  hard¬ 
ware  and  computer  software  will  be  spec¬ 
ified  and  treated  as  configuration  items. 

Baseline  implementation  guidance  for 
this  action  is  contained  in  DoD  Instruc¬ 
tion  5010.21  (DoDD  5000.29,  1976*2). 

This  directive  is  not  without  conflict,  however.  For 
example,  the  use  and  definition  of  related  terms,  such  as 
computer  software,  computer  data,  computer  program,  and 
software  engineering  have  been  interpreted* 

a.  as  indeed  sufficiently  loose  that  their  real  intent 
is  ambiguous  in  a  number  of  respects,  and 

b.  such  that  they  may  well  prove  to  be  in  significant 
conflict  with  established  Air  Force  practice 
(Searle,  1977*115). 

DoDD  5000.19.  This  directive,  entitled  "Configuration 
Management,"  establishes  policy  for  configuration  management 
for  all  Military  Departments,  DoD  components  at  all  eche¬ 
lons,  and  all  Defense- Industry  interfaces.  In  summary,  con¬ 
figuration  management  must  be  applied  to  all  configuration 
items  procured,  or  obtained  by  agreement  between  in-house 


52 


activities.  The  directive  describes  responsibilities  for 
initiation,  planning,  documentation  and  audit  of  configura¬ 
tion  management.  In  addition,  the  processes  of  functional 
and  allocated  configuration  identification,  control,  and 
status  accounting  are  described  (Glore,  ^t  aJL. ,  1978»44-45). 

DoDI  5010.21.  DoD  Instruction  5010.21,  "Configuration 
Management  Implementation  Guidance,"  provides  guidance  for 
DoD  components  in  the  implementation  of  Department  of  De¬ 
fense  policies  on  configuration  management.  It  specifies 
that  configuration  management  applies  to  all  systems,  equip¬ 
ment,  and  other  designated  material  items.  More  specifi¬ 
cally,  the  process  should  be  tailored  to  the  particular  con¬ 
figuration  item  whether  it  is  developed  at  government  ex¬ 
pense  or  privately  developed  and  offered  for  government  use. 
In  addition,  the  instruction  addresses  and  defines  guide¬ 
lines  related  to  the  following  concepts  of  configuration 
management i 

a.  configuration  identification, 

b.  configuration  control, 

c.  configuration  status  accounting, 

d.  configuration  audits, 

e.  procurement  aspects, 

f.  logistic  support  aspects,  and 

g.  implementation  (Glore,  et  al. ,  1978i46). 

Policies,  regulations,  and  procedures  implementing  the  first 
of  these  concepts  are  a  major  concern  of  this  thesis. 
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Therefore,  the  directives  that  address  the  process  of  al¬ 
locating  system  level  requirements  to  CPCIs  are  summarized 
in  the  following  sections. 

LilL  Force  Regulations 

Recent  emphasis  in  computer  software  acquisition  has 
prompted  the  publication  of  numerous  policies,  regulations, 
standards,  and  guidebooks  in  the  Air  Force.  Some  of  these 
have  increased  the  emphasis  that  managers  and  users  of  sys¬ 
tems  must  place  on  the  computer  resources.  Others  have 
clarified  and  broadened  organizational  responsibilities  and 
authority.  For  example,  using  and  supporting  commands  are 
required  to  accept  an  early  role  in  planning  for  the  opera¬ 
tional  and  support  requirements  of  software.  However,  the 
success  of  new  policies  rests  not  on  their  publication,  but 
on  a  full  understanding  of  the  concepts  underlying  them  and 
their  proper  implementation.  Therefore,  the  most  signifi¬ 
cant  directives  affecting  the  acquisition  and  management  of 
software  are  summarized  below.  Particular  emphasis  is 
placed  on  those  establishing  policies  and  procedures  for 
CPC  I  selection. 

AFR  800-2.  This  regulation,  entitled  "Acquisition  Pro¬ 
gram  Management,"  implements  DoDD  5000.1  and  DoDD  5000,2  by 
establishing  policy,  responsibilities,  and  reporting  re¬ 
quirements  for  major  systems  and  for  other  Air  Force  ac¬ 
quisition  programs.  Maximum  authority  and  responsibility 
for  each  program  are  delegated  to  the  implementing  command. 
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In  addition,  a  single  person  known  as  the  Program  Manager 
<PM)  is  directed  to  plan,  organize,  and  conduct  the  acquisi¬ 
tion  within  the  Air  Force  approved  limits  of  system  per¬ 
formance,  schedule,  and  funding.  Therefore,  the  Program 
Manager  is  the  technical  and  administrative  focal  point  for 
all  program  activities,  including  the  participation  of  all 
other  organizations  (AFR  800-2,  1 977 » 1 -2 ) . 

AFR  800-14.  The  most  important  regulation  affecting 
ECS  software  acquisition  is  AFR  800-14,  Volumes  I  and  II. 
Along  with  Air  Force  Systems  Command  supplement  1  to  Vol¬ 
ume  I,  this  regulation  establishes  policy  and  provides  guid¬ 
ance  for  planning,  developing,  acquiring,  supporting,  and 
using  computer  resources  in  systems  acquired  and  managed 
under  the  AFR  800-2  program  management  concept.  Thus,  com¬ 
puter  resources  are  the  only  commonly  used  components  of  Air 
Force  weapon  systems  whose  development  is  addressed  in  a 
separate  regulation.  This  is  partially  because  computer 
technology  is  relatively  new  and  not  well  understood.  In 
addition,  software  is  on  the  critical  path  of  many  procure¬ 
ment  efforts,  and  is  therefore  the  root  cause  of  many  costly 
delays  in  system  acquisitions. 

Volume  I,  entitled  "Management  of  Computer  Resources  in 
Systems,"  establishes  policy  for  computer  resources  employed 
as  dedicated  elements,  subsystems,  or  components  of  systems 
acquired  and  managed  under  AFR  800-2.  More  specifically, 
the  Air  Force  policy  on  management  of  computer  resources  in 
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systems  states  thati 


a.  Computer  resources  in  systems  are  managed  as  ele¬ 
ments  or  subsystems  of  major  importance  during  con¬ 
ceptual,  validation,  full-scale  development,  pro¬ 
duction,  employment,  operation,  and  support  phases. 
System  performance  requirements  are  allocated  to 
subsystems  using  in-depth  trade-off  studies  and 
cost-effectiveness  analyses, 

b.  Configuration  management  procedures  are  developed 
to  assure  control  during  development,  test,  trans¬ 
fer  and  turnover,  operational  maintenance,  and 
major  modif ication. 

c.  Program  Management  Direc tives ( PMDs )  require  and 
Program  Management  Plans (PMPs)  provide  for  the 
specification  and  allocation  of  system  performance 
and  interface  requirements  to  be  met  by  computer 
equipment  and  computer  programs. 

d.  PMDs  require  and  PMPs  provide  for  the  identifica¬ 
tion  of  computer  equipment  and  computer  programs 
as  configuration  items. 

(AFR  800-1^,  Vol.  I,  1975il-2) 
In  addition,  Volume  I  assigns  implementation  responsibili¬ 
ties  to  Headquarters  U.S.  Air  Force,  Air  Force  Systems  Com- 
mand(AFSC),  Program  Manager,  Air  Force  Logistics  Command 
(AFLC),  Using  agencies,  Air  Training  Command (ATC)  and  the 
Air  University  (AFR  800-14,  Vol.  I.  1975*2-3). 

Volume  II  of  AFR  800-14,  entitled  "Acquisition  and  Sup¬ 
port  Procedures  for  Computer  Resources  in  Systems,"  consoli¬ 
dates  procedures  that  apply  when  implementing  the  policies 
of  AFR  800-14,  Volume  I  and  other  related  publications.  It 
provides  a  centralized  source  document  for  the  policies,  pro¬ 
cedures,  and  guidance  required  to  manage  the  acquisition  and 
support  of  computer  resources  in  systems.  The  basis  for  this 
volume  is  "the  uniqueness  of  computer  resource  management 
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requires  the  publication  of  a  document  which  consolidates 
and  explains  the  applicability  of  other  publications  to  com¬ 
puter  resource  acquisition  and  support"  (AFR  800-14,  Vol. 

II.  1975*1-1 ). 

The  specific  areas  this  regulation  addresses  are  as 
follows  I 

a.  Computer  resources  in  the  system  acquisition  life 
cycle. 

b.  Planning. 

c.  Engineering  management. 

d.  Computer  program  quality  assurance. 

e.  Configuration  management. 

f.  Documentation. 

g.  Contractual  aspects. 

h.  Transition  ard  turnover. 

i.  Support. 

j.  Computer  program  validation  and  verif ication(VAV) . 

(AFR  800-14,  Vol.  II,  1975) 

Since  the  "other  related  publications"  (referred  to  above) 
that  are  applicable  to  this  thesis  are  summarized  in  later 
paragraphs,  a  detailed  description  of  each  area  is  not  in¬ 
cluded  here.  However,  the  computer  program  life  cycle  de¬ 
fined  in  this  regulation  is  described  since  it  varies  some¬ 
what  with  that  normally  found  in  textbooks  and  other  non¬ 
government  publications.  In  addition,  the  interrelation  of 
the  system  acquisition  life  cycle  and  the  computer  program 
life  cycle  are  described  below. 
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Computer  program  development  can  be  conceptualized  as 
the  computer  program  life  cycle  shown  in  Figure  4  (AFR  800- 
14,  Vol.  II,  1975»2-5).  The  primary  difference  between  this 
and  the  non-government  definition  is  the  Air  Force  treatment 
of  software  installation  as  a  separate  phase.  The  rationale 
for  this  appears  to  be  the  need  to  place  special  emphasis  on 
the  peculiar  adaptation  of  the  software  to  various  sites  for 
multi-site  systems.  In  other  words,  the  computer  programs 
should  be  demonstrated  to  operate  with  a  specified  level  of 
confidence  at  each  site  although  they  have  previously  been 
successfully  qualified  and  integrated. 

The  interrelationships  of  the  system  acquisition  life 
cycle  and  the  computer  program  life  cycle  can  vary  depending 
on  the  specific  program  under  development.  The  computer 
program  life  cycle  may  span  more  than  one  system  acquisition 
life  cycle  phase  or  occur  in  any  one  phase.  For  example,  a 
mission  simulation  computer  program  may  undergo  all  of  the 
phases  of  the  computer  program  life  cycle  during  the  concep¬ 
tual  phase i  while  a  mission  application  program  may  undergo 
the  computer  program  life  cycle  phases  during  the  validation, 
full-scale  development,  and  production  phases.  In  either 
case,  the  computer  program  life  cycle,  and  the  formal  activ¬ 
ities  associated  with  it  (configuration  management,  techni¬ 
cal  reviews,  testing,  audits,  etc.),  will  occur  at  least 
once  for  each  CPC  I  during  the  system  acquisition  life  cycle. 
The  activities  need  not  be  sequential!  instead,  there  are 
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Computer  Program 


potential  loops  between  all  phases.  For  example,  design  may 
reveal  problems  which  lead  to  the  revision  of  requirements 
and  reinstitution  of  certain  analyses  (APR  800-14,  Vol,  II, 
1975t2-3). 

The  DoD  Directives  and  Air  Force  Regulations  covered  to 
this  point  have  been  primarily  concerned  with  the  acquisition 
and  management  of  computer  resources  in  systems.  They  were 
summarized  to  establish  the  overall  policies  on  computer 
programs.  Within  that  general  framework,  the  specific  pol¬ 
icies,  concepts,  techniques,  and  procedures  for  CPCI  selec¬ 
tion  on  major  system  acquisitions  (as  prescribed  by  APR 
65-3.  and  AFSC  Pamphlets  8OO-3  and  800-7)  are  described 
below. 

APR  65-3.  This  joint  DoD  Services/Agency  regulation, 
entitled  "Configuration  Management,"  prescribes  uniform  pol¬ 
icies  and  guidance  for  the  Military  Services  and  Defense 
Agencies  (hereafter  referred  to  as  DoD  components)  respon¬ 
sible  for  implementation  of  configuration  management  within 
the  DoD.  The  purpose  of  conf iguration  management  is  to 
identify,  control,  account  for,  and  audit  the  functional  and 
physical  characteristics  of  systems,  equipments,  and  other 
designated  material  items  developed,  produced,  operated,  and 
supported  by  DoD  components.  Although  the  provisions  of  this 
regulation  apply  to  all  DoD  components,  it  is  not  intended 
to  impose  strict  unalterable  rules  on  the  implementing 
agency.  Rather,  the  configuration  management  process  is  to 
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be  carefully  tailored  to  the  quantity,  size,  scope,  stage  of 
acquisition  life  cycle,  nature,  and  complexity  of  the  Cl  in¬ 
volved,  whether  the  Cl  is  developed  at  Government  expense  or 
privately  developed  and  offered  for  Government  use  (APR  65- 
3.  1975*1-1 ). 

APR  65-3  further  states  thati 

The  selection  of  CIs  to  be  configuration- 
managed  is  determined  by  the  Governments* 
need  to  control  a  Cl's  inherent  character¬ 
istics  or  to  control  that  Cl's  interface 
with  other  items.  The  selection  of  prime 
and  lower  level  CIs  is  basically  a  man¬ 
agement  decision  normally  accomplished 
through  the  system  engineering  process. 

The  decision  is  based  on  numerous  engi¬ 
neering/logistic  factors  (AFR  65-3, 

1975*1-1). 

However,  configuration  identification  is  applied  to  all 
hardware  and  computer  programs.  This  identification  is  also 
the  basis  for  the  preparation  of  technical,  administrative, 
and  management  documents.  This  documentation  includes  spe¬ 
cifications,  work  breakdown  structures,  technical  reports, 
test  plans,  test  procedures,  test  reports,  cost  performance 
reports,  and  provisioning  documents.  In  addition,  config¬ 
uration  identification  is  the  basis  for  configuration  con¬ 
trol  and  status  accounting  during  the  life  cycle. 

Configuration  identification  is  defined  as  the  process 
of  documenting  performance,  qualification,  fabrication,  and 
acceptance  requirements  (AFSCP  800-7,  1977*16).  The  docu¬ 
mented  requirements  are  called  a  "baseline"  which  is  im¬ 
plemented  by  multilateral  agreements  among  the  contractor, 
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procuring  agency,  and  user.  Three  common  baselines  (func¬ 
tional,  allocated,  and  product)  are  described  in  the  aggre¬ 
gate  of  specifications,  drawings,  parts  lists,  computer 
listings,  and  other  documentation  used  to  describe  the  func¬ 
tional  and  physical  characteristics  of  a  system.  These 
baselines  and  their  relationship  to  the  system  acquisition 
life  cycle  are  depicted  in  Figure  5  (AFSCP  800-3,  1976 »9-3). 
However,  configuration  identification  is  not  necessarily 
synonymous  with  configuration  baselines i  unbaselined  iden¬ 
tification  is  used  for  visibility,  and  baselines  are  used 
for  control.  Identification  constitutes  a  baseline  only 
when  formally  designated  and  fixed  at  a  specific  point  as  a 
reference  point  for  change  control  (AFSC  Supplement  1,  APR 
65-3.  1975*1). 

In  addition  to  the  above  definitions  and  guidelines, 

AFR  65-3  prescribes  implementing  instructions  for  Air  Force 
organizations.  The  policies  and  procedures  most  relevant  to 
this  thesis  are  delineated  be low t 

a.  Computer  programs  will  be  managed  as  essential 
system  elements  using  common  procedures  tailored 
to  recognize  their  unique  properties. 

b.  For  each  Cl,  an  individual  will  be  assigned  the  re¬ 
sponsibility  and  delegated  the  authority  for  man¬ 
agement  of  the  item's  configuration. 

c.  AFSC  will  select,  in  collaboration  with  AFLC  or 
United  States  Air  Force  Security  Service (USAFSS) 
and  the  Using  command(s),  the  configuration  items 
on  which  configuration  status  accounting  will  be 
initiated  and  maintained. 
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d.  AFLC  and  U3AF33  will,  for  material  for  which  each 

is  responsible,  assist  AF3C  in  the  selection  of  con¬ 
figuration  items  to  be  managed  and  monitor  their 
early  development  for  logistics  planning. 

e.  Configuration  management  will  be  applied  to  each 
CPCI  throughout  the  system  acquisition  life  cycle. 

(AFR  65-3.  1975«F-1) 

AFSCP  800-7.  This  pamphlet,  entitled  "Configuration 
Management,"  supersedes  AFSC/AFLC  Manual  375-7  and  contains 
philosophy,  methods,  and  procedures  based  on  DoD  and  Air 
Force  policy.  It  is  intended  to  be  a  tutorial  text  as  well 
as  a  reference  for  those  who  are  working  in  conf iguration 
management  daily.  More  specifically,  the  pamphlet  states 
that  i 


The  actual  application  should  be  guided 
by  AFR  65-3/AFJC  Sup.  1  and  the  judge¬ 
ment  and  experience  of  the  Program  Man¬ 
ager  and  configuration  management  office 
(AFSCP  800-?,  1977*5). 

Therefore,  some  flexibility  is  permitted  in  the  decomposition 
of  system  requirements  (defined  in  a  system  on  system-seg¬ 
ment  specification)  into  lower  level  requirements  that  are 
subsequently  specified  in  CPCI  prime  item  development  speci¬ 
fications.  This  flexibility  is  based  on  the  idea  that  CIs 
usually  cannot  be  definitely  selected  before  final  design  of 
the  system.  In  fact,  this  pamphlet  states  that* 

The  initial  Cl  list  should  contain  more 
items  than  is  anticipated  on  the  final 
list.  Deletion  of  an  item  is  preferable 
to  backtracking  after  production  to 
build  a  data  bank,  since  retrieval  of 
data  after  the  fact  is  difficult  and 
costly  (AFSCP  800-7,  1977*31). 


64 


Although  the  selection  of  CPCIs  is  not  an  exact  science,  it 
should  be  approached  in  the  same  manner  as  hardware  Cl  se¬ 
lection,  This  leads  to  several  criteria  for  use  in  selecting 
CPCIs. 

This  pamphlet  specifies  that  while  one  major  governing 
factor  is  that  a  Cl  should  have  characteristics  that  make  it 
feasible  to  be  produced  by  a  single  contractor  and  tested  as 
an  entity j  another  very  important  factor  is  the  level  at 
which  there  is  a  capability  to  manage.  In  other  words,  if 
too  many  CIs  are  established  within  a  system,  control  may  be 
lost.  Therefore,  in  designing  a  system  and  selecting  CIs, 
the  management  resources  of  Government  program  offices  and 
contractors  must  be  considered  (AF3CP  800-7,  1977i9).  In 
addition,  separate  CPCIs  should  be  initially  established  for 
generic  categories  (e.g.,  operational,  test,  and  support 
computer  programs).  A  further  division  may  be  made  for  con¬ 
tract  or  subcontract  purposes,  for  logistics  management  and 
operation,  or  to  match  up  the  system  confix  ^tlon  with  the 
statement  of  work  and  work  breakdown  structure  (AFSCP  800-7, 
1977i76). 

These  general  guidelines  are  also  supplemented  with 
more  definitive  criteria  for  CPC  I  selection.  They  are  ba~pd 
on  the  Air  Force  policy  that  CPCIs  satisfy  an  end  use  func¬ 
tion  and  are  designated  by  the  procuring  activity  as  subject 
to  configuration  management  procedures.  Specifically,  they 
are  as  follows* 


a.  Assign  processes  that  interact  strongly  (e.g.,  in 
many  or  complex  ways)  to  the  same  CPC I. 

b.  Assign  processes  with  little  or  no  interaction  to 
different  CPCIs. 

c.  Allocate  to  different  CPCIs  processes  that  will 
execute  in  different  computers. 

d.  Assign  to  different  CPCIs  processes  whose  develop¬ 
ment  can  feasibly  be  finished  at  significantly  dif¬ 
ferent  times,  if  such  phased  development  will  ex¬ 
pedite  overall  system  development. 

e.  Allocate  to  different  CPCIs,  computer  programs  to 
be  separately  procured. 

f.  Include  in  each  CPC  I  no  more  than  an  individual 
Government  monitor  can  efficiently  track,  assuming 
reasonable  working  relationships  between  him  and 
the  types  of  personnel  who  will  manage  and  develop 
the  CPC I. 


( AF3CP  800-7,  1977 i76) 

In  summary,  the  selection  of  CPCIs  is  one  of  the  most 
critical  decisions  made  during  the  acquisition  of  a  major 
defense  system.  They  provide  the  structure  for  technical 
direction,  administrative  actions,  management  decisions, 
visibility,  documentation,  and  contractual  delivery.  There¬ 
fore,  the  criteria  used  for  selecting  CPCIs  requires  careful 
consideration  by  the  contractor  and  System  Program  Office 
(SPO). 

To  emphasize  the  importance  of  this  decision,  and  con¬ 
figuration  management  in  general,  the  Statement  Of  Work (SOW) 
and  contract  for  acquisition  programs  are  tailored  to  the 
specific  requirements  of  the  system.  AFSCP  800-7  also  con¬ 
tains  guidelines  and  instructions  for  applying  configuration 

management  contractually.  Some  of  the  instructions  and 

66 


sample  clauses  that  have  been  used  in  Request  for  Proposals 
(RFPs)  are  provided  in  this  pamphlet.  Those  relevant  to  this 
research  project  are i 

a.  Configuration  identification  should  be  established 
in  the  form  of  specifications  and  referenced  tech¬ 
nical  documentation.  This  identification  becomes 
more  detailed  as  design  and  testing  progresses. 

b.  The  Allocated  Configuration  Identif ication(ACI)  is 
documented  by  a  performance  oriented  specification 
in  accordance  with  KIL-STD-490,  KIL-3TD-483,  and 
MIL-S-83490.  This  identification  documents  func¬ 
tional  requirements  allocated  to  a  Cl  from  a  system 
or  higher  level  Cl  and  is  contained  in  Cl  develop¬ 
ment  specifications.  The  AC  I  is  formally  estab¬ 
lished  during  Advance  Development/Validation  or 
Full-Scale  Development  for  the  CIs. 

AFSCP  800-3.  AF3C  Pamphlet  800-3  also  establishes  pol¬ 
icies  and  implementing  guidance  governing  the  configuration 
management  of  systems,  equipments,  and  computer  programs. 

In  most  cases,  the  requirements  are  consistent  with  pre¬ 
viously  cited  directives.  However,  this  pamphlet,  entitled 
"A  Guide  for  Program  Management,  **  does  perpetuate  the  con¬ 
fusion  and  conflict  on  whether  software  should  be  considered 
an  item  of  data  or  an  active  system  component.  The  guide¬ 
line  specifically  states  1 

For  this  guide  and  clarity,  contractor 
prepared  data  acquired  on  DD  Form  1423. 

Contract  Data  Requirements  List,  as  a 
part  of  System/Equipment  Procurements 
are  essentially  anything  other  than 
hardware  as  depicted  by  Figure  6  (AFSCP 
800-3,  l976«l6-2). 

This  policy  is  based  on  the  requirements  of  the  ASPR,  Sec¬ 
tion  IX  and  DPC  Circular  74-3  which  states  that  computer 

•  ftware  falls  within  the  definition  of  data.  Although  AFSC 
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has  indicated  that  computer  software  acquisition  should  be 
subjected  to  the  same  contractual  and  cost  controls  as  hard¬ 
ware  (AFSCRP  800-1,  19750).  confusion  will  abound  until 
this  directive  and  the  ASPR  are  changed. 

The  selection  of  configuration  items  is  also  addressed 
in  this  pamphlet.  Several  factors  are  cited  as  the  basis 
for  making  the  selection  decision.  Those  applicable  to  CPCI 
selection  are i 

a.  Each  system  unit  is  designed  to,  and  controlled  by, 
one  Cl  specification. 

b.  Each  computer  program  is  identified  and  documented 
by  one  top  flow  chart  and  a  structure  of  subordi¬ 
nate  flow  charts  and  detailed  descriptions. 

c.  However,  in  the  final  analysis,  selecting  CIs  re¬ 
quires  the  exercise  of  certain  judgements  based  on 
experience  and  program  requirements.  No  set  of 
rules  can  adequately  describe  this  judgement 
factor. 

d.  While  the  selection  of  CIs  on  a  program  is  usually 
made  through  the  cumulative  recommendations  from 
several  sources,  it  is  also  an  area  where  early  de¬ 
cisions  can  benefit  the  program.  This  is  not  to 
say  that  CIs  can  be  definitely  selected  before 
final  design  of  the  system,  for  flexibility  is  a 
necessity.  It  would  be  desirous  for  a  preliminary 
list  to  be  compiled  as  early  as  possible. 

e.  It  is  very  important  to  include  the  AFLC  System 
Manager(SM)  in  the  process.  With  the  advent  of  new 
weapon  systems,  it  is  expected  that  they  will  be 
selected  by  AFLC  for  application  of  the  AFLC  Ad¬ 
vanced  Configuration  Management  System.  In  most 
cases,  SM  selections  will  para^el  those  made  by 
the  SPO  and  again,  cost  savings  can  be  realized  if 
these  are  resolved  in  the  early  program  stages. 

( AFSCP  800-3,  1976i9-^) 
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Mill  tarv  Standards 


Military  Standards  covering  configuration  management  re¬ 
quirements  and  procedural  guidance  are  the  basic  documents 
for  contractually  implementing  a  configuration  management 
program  on  acquisition  projects.  They  provide  guidelines  to 
both  government  and  contractor  personnel  responsible  for  one 
of  the  various  aspects  of  configuration  management.  One  of 
these,  MIL-STD-483  (USAF),  entitled  "Configuration  Management 
Practices  for  Systems,  Equipment,  Munitions,  and  Computer 
Programs,"  establishes  uniform  configuration  management 
practices  that  can  be  tailored  to  all  USAF  systems.  in 
other  words,  contracts  invoking  this  standard  may  identify 
the  appropriate  paragraphs  and  appendices  as  applicable  to 
that  acquisition  project. 

Appendix  XVII  of  this  standard  specifies  that  the  se¬ 
lection  of  CIs  is  based  on  the  definition  contained  in  DoDD 
5000.19,  i.e.,  a  Cl  is  "an  aggregation  of  hardware/software, 
or  any  of  its  discrete  portions,  which  satisfies  an  end  use 
function  ..."  Furthermore,  Cl  selection  reflects  an  optimum 
management  level  during  acquisition.  This  level  is  one  at 
which  the  procuring  activity  specifies,  contracts  for,  and 
accepts  individual  elements  of  a  system  (MIL-STD-483,  Notice 
2,  1979*129). 

Several  general  and  specific  considerations  are  pre¬ 
sented  in  this  recently  published  appendix,  entitled  "A 
Guide  for  Selecting  Conf iguration  Items. "  Most  of  the 


general  guidelines  are  covered  in  previously  summarized  di¬ 


rectives.  However,  the  following  ones  deserve  specific 
mentions 


a.  Selecting  CIs  should  be  with  a  full  view  of  the 
life  cycle  cost  and  management  impacts  associated 
with  such  a  designation.  Choosing  too  many  CIs 
increases  the  cost  of  controls  choosing  too  few  or 
the  wrong  elements  as  CIs  runs  the  risk  of  too 
little  control  through  lack  of  management  visi¬ 
bility. 

b.  The  major  elements  comprising  the  system  should  be 
identified  as  CIs  during  the  Demonstration/Valida¬ 
tion  Phase.  Early  selection  of  CIs  is  important 
since  management  emphasis  becomes  greater  as  de¬ 
velopment  progresses.  As  development  continues  and 
logistic  and  technical  considerations  surface,  ad¬ 
ditional  items  can  be  designated  as  CIs.  Usually, 
the  Cl  selection  process  should  be  essentially  com¬ 
plete  by  the  Preliminary  Design  Review. 

(MIL-STD-483,  Notice  2,  1979il29-130) 
Some  of  the  specific  considerations  upon  which  the  selection 
decision  shall  be  based  are i 

a.  This  selection  is  usually  a  technically  driven  de¬ 
cision  made  by  the  developer.  It  i3  based  upon 
system  trade-offs  and  the  natural  decomposition  of 
the  software.  Premature  partitioning  must  be 
avoided  because,  to  a  certain  extent,  it  may  pre¬ 
ordain  the  design. 

b.  Generally,  the  executive /supervisor ,  functional/ 
applications,  input/output,  test,  and  support  pro¬ 
grams  should  be  individual  CPCIs.  Computer  programs 
with  potential  use  in  multiple  systems  should  be 
separate  CPCIs.  Highly  interrelated  computer  pro¬ 
grams  should  be  combined  as  one  CPCI.  CPCIs  should 
be  established  to  their  largest  functional  element 
(i.e.,  Operational  Flight  Program,  Flight  Simulator 
Computer  Program,  etc.). 

c.  Assemblies  of  computer  program  elements  to  be  ac¬ 
quired  from  a  single  contractor  are  potentially  a 
single  CPCI. 
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d.  Computer  programs  to  be  designated  for  operation  in 
different  models  of  computers  should  be  separate 
CPC  Is. 

e.  Computer  programs  scheduled  for  development,  test¬ 
ing,  and  delivery  at  different  times  may  be  sepa¬ 
rate  CPC  Is. 

(MIL-STD-483,  Notice  2,  1979 «1 31-1 32 ) 

As  previously  indicated,  the  effects  of  improper  CPC  I 
selection  can  be  devastating  to  an  acquisition  program.  All 
three  evaluation  parameters  (cost,  schedule,  and  perfor¬ 
mance)  for  the  government,  prime  contractors,  sub-contrac¬ 
tors,  and  suppliers  can  be  adversely  affected.  A  detailed 
listing  of  the  usual  effects  of  Cl  desigantion  is  included 
in  this  appendix  to  MIL-STD-483.  In  addition,  &  series  of 
questions  which  makes  up  a  checklist  to  be  used  in  selecting 
CIs  is  provided.  According  to  this  standard,  the  system 
component  probably  should  not  be  a  Cl  if  most  of  the  ques¬ 
tions  can  be  answered  "no."  Likewise,  if  most  of  the  ques¬ 
tions  can  be  answered  "yes,"  the  item  should  be  a  Cl.  Addi¬ 
tional  judgement  is  needed  if  the  questions  can  be  answered 
with  approximately  equal  numbers  of  "yes"  and  "no,"  Since 
this  thesis  concerns  the  criteria  for  CPC  I  selection,  the 
checklist  is  reproduced  below « 

a.  Is  it  a  critical  high  risk,  and/or  a  safety  item? 

b.  Is  it  readily  identifiable  with  respect  to  size, 
shape,  and  weight  (hardware)? 

c.  Is  it  newly  developed? 

d.  Does  it  incorporate  technologies? 

e.  Does  it  have  an  interface  with  hardware  or  s<  *  s»are 
developed  under  another  contract? 
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f.  With  respect  to  form,  fit,  or  function,  does  it 
interface  with  other  items  whose  configuration  is 
controlled  by  other  entities? 

g.  Is  there  a  requirement  to  know  the  exact  configura¬ 
tion  and  status  of  changes  to  it  during  its  life 
cycle? 


(MIL-STD-483,  Notice  2,  1979»134) 


Chanter  Summary 

The  acquisition  and  management  of  computer  resources 
within  the  Federal  Government  is  controlled  by  a  complex 
hierarchy  of  policies  and  directives.  In  general,  those 
applicable  to  the  acquisition  and  management  of  computer  re¬ 
sources  during  the  development,  acquisition,  deployment,  and 
suppo'  i  of  major  defense  systems  were  summarized.  More  spe¬ 
cifically,  the  DoD  and  AF  policies,  procedures,  and  guide¬ 
lines  for  the  allocation  of  system  software  requirements  to 
CPC  Is  on  a  large  s of tware- in tensive  system  were  emphasized. 
Unfortunately,  only  a  few  of  the  documents  address  the  se¬ 
lection  of  CPCIs  directly.  On  the  other  hand,  several  do 
present  guidelines  and  considerations  for  applying  configu¬ 
ration  management  concepts  to  the  acquisition  and  management 
of  software. 

At  the  top  of  the  regulatory  structure  governing  the 
acquisition  and  management  of  computer  resources  is  Public 
Law  89-306.  This  law,  called  the  "Brooks  Bill,"  provides 
the  basic  structure  and  concepts  for  the  government-wide 
system  of  ADPE  management.  OMB  Circular  74-5  specifically 
excluded  those  computer  resources  integral  to  weapon  systems 


and  specially  desired  computer  equipment  from  control  of 
the  centralized  procurement  concept  of  P.L.  89-306.  This 
exception  was  implemented  via  OMB  Circular  A-109,  DoD  Di¬ 
rectives  5000.1,  5000.2,  and  5000.29,  the  Air  Porce  800- 
series  regulations  and  the  Air  Force  Systems  Command  800- 
series  pamphlets. 

In  essence,  these  documents 

a.  define  "major  system  acquisitions"  and  provide  the 
policies,  procedures,  and  guidelines  for  their  man¬ 
agement, 

b.  establish  policy  for  the  management  and  control  of 
computer  resources  that  are  part  of  a  major  system 
acquisition, 

c.  require  that  computer  resources  be  managed  as  ele¬ 
ments  or  subsystems  of  major  importance  during  the 
system  acquisition  life  cycle,  and 

d.  specify  that  both  computer  hardware  and  software 
will  be  treated  as  configuration  items. 

The  requirement  for  applying  configuration  management 
concepts  to  software  led  to  the  development  of  several  other 
policies  and  regulations.  Those  applicable  to  the  selection 
of  CPC  Is  are  DoD  Directive  5000.19,  DoD  Instruction  5010.21, 
AF  Regulation  65-3,  AFSC  Pamphlet  800-7,  and  Military  Stan¬ 
dard  483.  Although  these  documents  in  general  establish  uni¬ 
form  policies  and  procedures  the  implementation  of  con¬ 
figuration  management  within  the  DoD,  the  actual  application 
to  software  is  often  a  management  decision  which  requires 
the  judgement  and  experience  of  the  Program  Manager.  For 
example,  the  selection  of  CPC Is  is  not  an  exact  science.  As 

a  result,  several  criteria  for  use  in  allocating  software 
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requirements  to  CPC  Is  are  presented  in  these  documents.  In 
summary,  they  sure  as  follows! 

a.  The  initial  Cl  list  should  contain  more  items  than 
is  anticipated  on  the  final  list. 

b.  The  Cl  should  be  produced  and  tested  by  a  single 
contractor  as  an  entity. 

c.  Assign  processes  that  interact  strongly  to  the  same 
CPCI. 

d.  Allocate  to  different  CPC  Is  processes  that  will 
execute  in  different  computers. 

e.  Assign  to  different  CPCIs  processes  whose  develop¬ 
ment  can  feasibly  be  finished  at  significantly  dif¬ 
ferent  times. 

f.  Include  in  each  CPCI  no  more  than  an  individual 
Government  monitor  can  efficiently  track. 

g.  Selecting  CIs  should  be  with  a  full  view  of  the 
life  cycle  cost  and  management  impacts  associated 
with  such  a  designation.  Choosing  too  few  or  too 
many  CIs  adversely  affects  the  program. 

h.  The  selection  of  CPCIs  is  usually  a  technically 
driven  decision  made  by  the  developer  as  a  result 
of  system  trade-offs  and  the  natural  decomposition 
of  the  software. 

One  other  significant  finding  of  this  review  was  the 
ASPR  treatment  of  computer  programs.  Considering  software 
as  an  item  of  data  in  the  same  manner  as  reports,  forms, 
manuals,  and  specifications  causes  confusion  and  conflict. 
AFSC  has  initiated  an  attempt  to  have  the  ASPR  committee  re¬ 
verse  its  decision.  In  the  meantime,  AFSC  has  continuously 
required  that  computer  software  be  subjected  to  the  same 
contractual  and  cost  controls  as  hardware. 
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IV.  Non_-Regulatory  Publications 


Over  the  past  few  years,  there  has  been  a  steadily 
sharpening  focus  on  what  has  been  characterised  as  the  "DoD 
software  problem. "  This  focus  is  evidenced  by  the  large 
number  of  non-regulatory  publications  addressing  software 
acquisition  and  management  problems.  This  has  stimulated 
senior  decision  makers,  most  of  whom  have  no  personal  ex¬ 
perience  with  software,  to  learn  about  it  and  give  it  manage¬ 
ment  attention  comraernsurate  with  the  $3  billion  DoD  spends 
on  it  every  year  (Carlson,  1976  079).  As  a  result,  the  DoD 
and  its  components  have  sponsored,  or  participated  in,  nu¬ 
merous  conferences,  workshops,  and  studies  for  the  purpose 
of  developing  a  better  understanding  of  the  so-called  "soft¬ 
ware  problem."  In  addition,  a  large  number  of  technical  re¬ 
ports  and  management  guidelines  have  been  either  prepared  or 
supported  by  the  DoD  and  its  agencies.  Despite  these  ef¬ 
forts,  some  weapon  system  acquisition  programs  are  still  ex¬ 
periencing  software  cost  growths  and  overruns,  schedule  de¬ 
lays,  and  unreliable  performance. 

There  is  a  considerable  body  of  reputable  sources  avail¬ 
able  which  document  these  undesirable  consequences.  It 
would  not  serve  the  purpose  of  this  thesis  to  summarize  each 
one  for  two  reasons.  First,  the  number  of  studies,  reports, 
and  lessons  learned  is  overwhelming.  Secondly,  the  specific 
topic  of  this  research  is  the  process  of  allocating  software 


requirements  to  computer  program  configuration  items  in  the 
acquisition  and  management  of  a  large  software-intensive 
weapon  system.  Therefore,  only  those  studies,  reports,  and 
lessons  learned  which  are  relevant  to  this  subject  are  con¬ 
sidered  in  this  chapter. 

Other  sources  reviewed  in  this  literature  search  in¬ 
clude  conference  proceedings,  textbooks,  periodicals,  and 
management  guidebooks.  In  most  cases,  these  sources  are 
directed  at  specific  functional  areas,  as  opposed  to  the 
problems  associated  with  CPC  I  selection  in  the  acquisition 
and  management  of  computer  resources.  Some  are  aimed  at 
identifying  problems  only,  while  others  address  both  prob¬ 
lems  and  solutions.  In  addition,  a  large  amount  of  "expert" 
opinion,  as  well  as  factual  material,  is  contained  in  these 
sources.  Therefore,  only  those  that  address  the  process  of 
CPC  I  selection  and  its  impact  on  the  overall  program  are 
summarized  below. 

DoD  Sponsored  Studies 

As  early  as  1964,  high-level  Air  Force  management  had 
already  recognized  that  computer  technology  was  developing 
rapidly  and  that  expanding  requirements  would  dictate  a  pro¬ 
liferation  of  applications.  It  was  predicted  (accurately) 
at  that  time  that  the  Air  Force  computer  inventory  would  in¬ 
crease  by  50  percent  in  the  ensuing  2  or  3  years.  That  pre¬ 
diction  was  followed  by  the  awarding  of  a  contract  to  the 
Planning  Research  Corporation  for  a  "Study  of  Application 

?? 


Effectiveness  and  Problems  of  Air  Force  Information  Process¬ 
ing  Systems."  Although  several  other  studies  addressed 
problems  with  the  computer  system  acquisition  process  in  the 
ensuing  11  years,  few  of  the  recommendations  were  ever  im¬ 
plemented.  In  summary,  the  various  studies  had  some  minimal 
effecti  however,  they  failed  to  achieve  the  positive  action 
required  to  solve  the  problems  (Drezner,  et  ai., ,  19?6«3). 

The  Computer  Resources  Management  Study.  In  early 
1975.  the  Air  Force  requested  that  The  Rand  Corporation  ex¬ 
amine  AF  management  of  computer  resources.  Basically,  the 
study  considered  the  broad  aspects  of  management,  policy, 
and  organization  relevant  to  the  computer  resources  for  ma¬ 
jor  weapon,  command  and  control,  telecommunication,  intelli¬ 
gence,  and  management  and  support  systems.  The  purpose  of 
this  study  was  not  only  to  identify  problems  but  to  propose 
alternative  courses  of  action  in  policy,  management,  and 
organization  and  to  identify  the  important  consequences  of 
implementation  of  the  alternatives.  The  only  finding/rec¬ 
ommendation  directly  related  to  the  requirements  allocation 
process  was  that  "the  requirements  process  is  exceedingly 
important,"  Furthermore,  "an  inadequate  ^('us  and  emphasis 
is  placed  on  the  requirements  process"  (Drezner,  Shulman, 
and  Ware,  1 975 « 5-8). 

The  MUDD  Report »  A  Case  Study  of  Navy  Software  Develop¬ 
ment  Practices.  This  report  is  the  result  of  a  year-long 
investigation  into  Navy  software  problems.  The  purpose  was 
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to  describe  some  of  the  problems  encountered  in  producing 
reliable  software  in  a  context  familiar  to  Navy  software 
developers  in  the  hope  that  they  could  recognize  and  avoid 
similar  errors  in  the  future.  A  mythical  software  system 
was  used  to  describe  where  and  how  the  developers  went  awry. 
The  pitfalls  outlined  typify  problems  which  actually  occurred 
in  software  developmental  efforts.  One  of  the  problems 
chronicled  in  this  report  was  stated  as  the  most  important 
design  question  confronting  the  system  designers.  That  is, 
"how  to  partition  the  system  in  such  a  way  that  it  would  be 
easy  to  modify."  Furthermore,  the  author  indicates  that 
project  managers  showed  a  lack  of  awareness  of  the  key  is¬ 
sues  involved  in  partitioning  software  when  questioned  on 
development  strategy  (Weiss,  1975*16). 

DoO  Weapon  Systems  Software  Acquisition  and  Management 
Study.  In  December  197^,  the  DoD  initiated  a  two-phased 
software  acquisition  study  program  to  identify  methods  for 
controlling  increasing  costs,  improving  the  quality,  and 
minimizing  the  adverse  impact  of  software  in  weapon  systems. 
The  MITRE  Corporation  and  the  Applied  Physics  Laboratory  of 
Johns  Hopkins  University  agreed  to  perform  separate,  but  co¬ 
ordinated,  four-month  studies  in  support  of  the  study  pro¬ 
gram,  In  Volume  I  of  MITRE's  report,  entitled  "Findings 
and  Recommendations,"  the  MITRE  Corporation  identified  sev¬ 
eral  major  systems  software  problem  areas  facing  the  DoD, 
their  causative  factors,  and  the  high  payoff  areas  that 
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should  be  given  priority  consideration  by  management.  In 
the  area  of  software  management  methods,  the  report  stated 
that  many  of  the  software  acquisition  and  management  prob¬ 
lems  can  be  traced  to  inadequate  requirements  formulation 
and  the  need  for  more  detailed  planning  during  the  early 
stages  of  weapon  system  acquisition  (Asch,  Kelliher,  Locher, 
and  Connors,  1975  a«2-7). 

To  recommend  a  course  of  action  for  improving  the  man¬ 
agement  and  control  over  costs,  the  quality,  and  the  timeli¬ 
ness  of  software  in  weapon  systems,  MITRE  extracted  from  the 
study  findings  the  areas  of  highest  payoff.  One  of  those 
areas  felt  to  have  the  greatest  leverage  is  the  Software 
Performance  Specification.  The  corrective  actions  in  this 
area  concern  the  recognition  and  consistent  application  of 
sound  engineering  principles  and  practices  to  the  process  of 
specifying  software  end  products,  i.e.,  CPCIs  (Asch,  et  al. . 
1975  ai2-13).  A  checklist  of  important  factors  to  be  applied 
across  all  systems  was  defined  and  recommended  to  be  the  sub¬ 
ject  of  required  DoD  guidelines.  Those  relevant  to  this  re¬ 
search  are  as  follows: 

a.  Choose  a  software  architecture  which  best  reflects 
the  weapon  system  requirements. 

b.  Emphasize  ease  of  change  in  the  software  perfor¬ 
mance  specification  process.  Recognize  that  weapon 
systems  software  requirements  will  change  over  the 
life  of  the  system.  Where  appropriate,  consider 
use  of  a  modular  architecture  which  allows  for 
changing  application  program  requirements. 

(Asch,  sJtal. ,  1975  a»3-7) 
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Volume  II  of  MITRE's  report,  entitled  "Supporting  Mate¬ 


rial,"  summarized  several  studies  and  workshop  proceedings. 
One  of  those ,  the  1973  Symposium  on  the  High  Cost  of  Soft¬ 
ware,  considered  what  research  is  needed  to  achieve  a  major 
reduction  in  software  costs.  Sponsored  by  the  Air  Force 
Office  of  Scientific  Research,  Army  Research  Office,  and 
Office  of  Naval  Research,  this  conference  was  organized  into 
five  workshops.  Each  workshop  was  assigned  a  specific  area 
to  investigate  and  develop  a  list  of  recommendations  on  how 
to  reduce  the  high  cost  of  software.  Workshop  5  dealt  with 
"problems  of  large  systems"  and  made  several  points  in  ad¬ 
dition  to  its  recommendations.  Four  of  these  points  con¬ 
cerned  the  design  and  structure  of  the  system.  They  arei 

a.  Premature  "freezing"  of  system  structure  into  man¬ 
agement  structure  causes  problems.  (Researcher's 
Note i  In  other  words,  it  is  very  difficult  to  over¬ 
come  the  inertia  that  naturally  develops  once  the 
management  structure  is  established.  As  a  result, 
efforts  are  duplicated,  system  integration  becomes 
more  difficult,  and  overall  visibility  may  be  lost 
as  the  system  design  evolves.) 

b.  Configuration  control  can  be  applied  too  early  and 
at  too  detailed  a  level  to  allow  for  continuing  de¬ 
sign  that  must  go  on. 

c.  The  procedures  usually  followed  in  government  con¬ 
tracted  software  are  not  flexible  enough  to  allow 
significant  tradeoffs  to  be  made  during  design  in  a 
timely  fashion. 

d.  Tight  configuration  control  should  be  delayed  until 
major  design  questions  have  been  answered. 

(Asch,  et  ai.,  1975  b»3-29,  3-31) 

Another  report  summarized  in  Volume  II  was  that  pre¬ 
pared  by  the  Army  Scientific  Advisory  Panel's  Ad  Hoc 
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Committee  for  Army  Tactical  Data  System  Software  Development. 
The  primary  objective  of  this  group  was  "to  determine  the 
exploratory  development  efforts  which  offer  the  best  promise 
of  satisfying  the  Army's  requirement  to  reduce  cost  and 
schedule,  and  increase  performance  and  reliability  of  soft¬ 
ware  for  major  Army  tactical  data  systems. "  One  of  the  com¬ 
mittee  *8  findings  was  that  "many  software  problems  are  root¬ 
ed  in  an  inadequate  initial  system  design  effort."  To  miti¬ 
gate  these  problems,  the  committee  recommended  that  an  order¬ 
ly  system  design,  considering  all  reasonable  alternative 
system  architectures,  be  performed  before  a  development  is 
initiated  (Asch,  et  al . ,  1975  b«3-6o). 

The  report  prepared  by  the  Applied  Physics  Laboratory 
of  the  Johns  Hopkins  University  summarized  their  findings 
and  presented  17  recommendations  directed  at  the  alleviation 
of  the  more  serious  software  problems.  One  of  the  recommen¬ 
dations  concerns  identifying  software  as  a  contract  deliv¬ 
erable,  Specifically,  the  recommendation  is  that  major  com¬ 
puter  software  involved  in  weapon  systems  development  be  des¬ 
ignated  as  CPC  Is  and  deliverables  during  Full  Scale  Develop¬ 
ment.  The  rationale  for  this  recommendation  was  that  the 
absence  of  clear  definitions  and  guidelines  applicable  to 
the  components  of  software  (computer  programs  and  computer 
data)  as  distinguished  from  software  documentation  has 
caused  numerous  instances  where  weapon  system  contracts  have 
not  called  for  appropriate  items  as  deliverables  (Kossiakoff, 
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Sleight,  Prettyman,  Park,  and  Hazan,  1 975 * 2-5 ) - 

Ballistic  Missile  Defense  Reliable  Software  Study.  In 
19?5.  the  Ballistic  Missile  Defense  Advanced  Technology  Cen¬ 
ter  (BMDATC)  commissioned  the  LOG ICON  Corporation  to  study 
current  BMDATC  research  efforts  and  develop  a  research  plan 
that  would  explore  basic  concepts  associated  with  reliable 
software.  More  specifically,  the  purpose  of  the  study  was 
to  develop  an  integrated  approach  that  would  allow  reliabil¬ 
ity  to  be  specified,  designed  in,  and  measured  during  soft¬ 
ware  development  (Ikezawa,  Kundu,  Lambert,  McGill,  and 
Nielsen,  1975*1-2).  Several  methods  for  performing  software 
requirements  analysis  were  presented  in  the  report.  One 
technique,  entitled  "Functional  Decomposition,"  was  dis¬ 
cussed  in  conjunction  with  the  top-down  design  process. 
Basically,  this  approach  produces  a  functional  model  of  the 
system  by  partitioning  the  requirements  into  threads,  where 
each  one  is  a  description  of  a  subsystem  function.  Then, 
each  thread  can  be  decomposed  into  the  tasks/modules  re¬ 
quired  to  perform  the  intended  function  (Ikezawa,  gjt  al. . 
1975*3-5).  However,  this  treatment  was  strictly  a  technical 
one.  In  other  words,  the  level  at  which  CPC Is  are  defined 
and  the  overall  impact  of  that  decision  on  the  acquisition 
and  management  of  the  system  was  not  discussed. 

Joint  Logistics  Commanders ( JLC )  Software  Reliability 
Work  Group (SRWG).  The  SRWG  was  originally  chartered  by  the 
JLC  to  develop  recommended  policies,  procedures,  and 


techniques  which  would  result  in  the  acquisition  and  fielding 
of  more  reliable  electronics  systems.  In  order  to  overcome 
the  difficulties  of  a  lack  of  common  understanding  of  the 
basis  for  software  management  practices  and  engineering 
technology,  a  multi-faceted  approach  was  undertaken  by  the 
SRWG.  For  example,  all  existing  DoD  regulations,  standards, 
and  directives  were  examined  for  applicability  to  software. 

In  addition,  all  ongoing  and  planned  technology  programs 
being  undertaken  by  the  various  DoD  agencies  were  reeval¬ 
uated  in  terms  of  current  problems  and  low  software  reli¬ 
ability  (Manley  and  Lipow,  1975«3)» 

From  these  investigations,  a  set  of  detailed  findings 
and  recommendations  for  the  solution  of  recognized  software 
problems  emerged.  The  first  finding,  entitled  "Software 
Management,  Policies,  and  Procedures,"  stated  that 

Existing  policies  governing  the  acquisi¬ 
tion  of  software  for  military  electronic 
systems  are  not  adequate  to  ensure  the 
delivery  of  reliable  software.  Although 
software  is  now  being  established  as  a 
configuration  item,  the  lack  of  suffi¬ 
cient  emphasis  in  existing  policies  on 
the  early  design  of  software  as  a  Cl 
tends  to  generate  reliability  and  other 
problems  during  subsequent  systems  pro¬ 
duction  and  operation  (Manley  and  Lipow, 

1975  «7). 

The  impact  statement  presented  in  support  of  this  finding 
stated  that 

The  reliability  of  operational  weapon 
system  software  is  seriously  degraded 
because  treatment  of  software  as  data 
rather  than  as  a  Cl  leads  to  a  lack  of 
focus  on  software  reliability  issues  and 


a  lack  of  appropriate  visibility  and  re¬ 
view  of  software  development  during  the 
acquisition  phase  (Manley  and  Lipow, 

1975«7/» 

Defense  System  Software  FY79-83  Research  and  Develon- 
ment(R&D)  Technology  Flan.  In  September  1977,  the  R&D  Tech¬ 
nology  Panel  to  the  Management  Steering  Committee  for  Em¬ 
bedded  Computer  Resources  in  the  Office  of  the  Director  of 
Defense  Research  and  Engineering  formulated  this  plan  to 
help  the  DoD  and  the  industry  managers  in  planning  and  eval¬ 
uating  their  R&D  initiatives  relating  to  software.  Several 
software  science  and  technology  areas  were  identified  for 
research.  In  addition,  projected  budget  profiles  for  the 
FY78-PY83  time  frame  were  specified.  One  of  the  areas  pro¬ 
jected  to  reveal  promising  results  is  Requirements  Analysis. 
More  specifically,  "Software  First"  technology,  competitive 
prototyping  guidelines,  semi-automated  requirements  analysis 
tools,  and  requirements  decomposition  are  planned  for  basic 
research  thrusts  (Carlson  and  DeRoze,  1977 ii,  1-1).  Al¬ 
though  the  plan  recognizes  that  conflicting  and  changing  re¬ 
quirements  will  always  be  a  problem,  it  establishes  the  need 
for  providing  insight  into  the  technical  implications  of 
stated  system  requirements  on  computer  resources  (and  vice 
versa),  identifying  risk  areas,  and  exploring  implementation 
alternatives  (Carlson  and  DeRoze,  1977»I-l). 

One rational  Software  Management  and  Development  for  U.S. 
Air  Force  Computer  Systems.  The  Committee  on  Operational 
Software  Management  and  Development  of  the  Air  Force  Studies 
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Board,  Assembly  of  Engineering,  National  Research  Council, 
conducted  a  study  of  the  problems  associated  with  the  ac¬ 
quisition  and  management  of  software  employed  in  AF  systems 
during  the  summer  of  1976.  The  purpose  of  the  study  was  not 
to  review  USAF  management  of  specific  systems,  but  rather  to 
examine  ways  the  USAF  might  become  more  effective  in  man¬ 
aging  and  developing  computer  resources  embedded  in  weapon 
systems.  The  study  primarily  focused  on  two  aspects  of 
software  management i  risk  reduction,  and  cost  and  schedule 
problems  (AF  Studies  Board,  1977 « 1-3)* 

The  committee  offered  seven  major  recommendations  and 
many  other  minor  ones  that  reinforce  and  extend  the  major 
ones.  Although  none  of  the  major  recommendations  directly 
addresses  the  selection  of  CPCIs,  the  second  one  would  im¬ 
pact  the  selection  decision  if  it  was  implemented.  In  es¬ 
sence,  the  committee  endorsed  "incremental"  development  by 
stating, 

"Experience  teaches  that  too  often  too 
much  is  attempted  at  once  in  a  develop¬ 
ment.  To  mitigate  this  problem,  we  rec¬ 
ommend  that  wherever  possible,  the  de¬ 
velopment  should  proceed  through  a  series 
of  deliveries.  Each  should  stand  by  it¬ 
self,  not  require  a  rework  of  its  pre¬ 
decessors,  and  each  should  be  of  tangible 
use.  During  this  period,  strong  feed¬ 
back  is  needed  from  the  using  organiza¬ 
tion  and  the  maintaining  organization  to 
the  development  activity,  in  order  to 
correct  any  recognized  problems"  (AF 
Studies  Board,  1977 ilO). 

In  other  words,  the  delivery  of  each  increment,  called  a 

"block,"  would  provide  a  complete  and  useful  functional 
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capability.  Two  major  reasons  were  stated  for  this  approach 
One  is  to  reduce  the  total  amount  of  code  to  be  developed  in 
the  first  block  and,  therefore,  to  get  the  core  parts  oper¬ 
ating  sooner.  Along  with  this  is  the  fact  that  such  a  strat 
egy  will  reinforce  the  need  for  a  sound  design  for  the  total 
program  and  will  better  force  independence  between  different 

•  application  modules  in  the  code  (AP  Studies  Board,  1977 136). 
Therefore,  the  CTCI  structure  of  the  software  could  be 
based,  at  least  in  part,  on  what  is  planned  for  each  block. 

Since  the  fifth  objective  of  this  research  was  to  de- 

4 

termine  the  feasibility  and  evaluate  the  potential  impacts 
of  defining  CPC  Is  in  terms  of  system  versions  or  models,  the 
committee's  findings  and  recommendations  on  the  incremental 
development  concept  are  briefly  summarized  as  follows  1 

a.  It  may  be  argued  this  approach  will  stretch  out  a 
project.  The  committee  thinks  not,  especially  not 
in  high  risk  projects,  where  problems  may  prevent 
almost  any  effective  delivery, 

b.  It  may  also  be  argued  that  breaking  some  programs 
into  blocks  is  artificial.  If  it  truly  is,  then 
the  committee  does  not  recommend  this  technique. 

c.  The  block  concept  should  also  permit  tradeoffs  in 
the  content  of  blocks.  This  will  accommodate  prob¬ 
lems  while  maintaining  the  overall  schedule.  How- 

1  ever,  to  guard  against  casual  or  arbitrary  slipping 

of  features  into  later  blocks,  an  incentive  fee 
contract  structure  that  rewards  the  contractor  for 
on-time  delivery  by  block  and  function,  and  denies 
rewards  when  features  or  blocks  are  delayed  is 

♦  recommended  by  the  committee.  The  incentive  fee 

is  a  major  tool  for  keeping  the  schedule  and  con¬ 
tent  in  each  block  in  agreement  with  the  contract’. 
But  it  is  still  only  a  tool  and  not  a  substitute 
for  informed  and  diligent  management  by  the  SPO. 
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d.  The  approach  is  compatible  with  DoD  Directive 
5000.29.  but  it  appears  that  USAF  Regulation  800-14 
needs  modification. 

e.  The  concept  is  of  general  applicability  and  should 
be  first  used  in  a  high  risk  project.  If,  then,  it 
is  successful,  the  committee  believes  review  will 
show  that  the  success  is  for  reasons  which  have 
more  general  applicability  than  just  for  high  risk 
projects. 

f.  Each  operational  program  should  be  a  01-01,  with 
block  I,  II,  III  deliveries  adding  up  to  one  of  the 
CPOIs  called  for  in  the  contract.  Software  tools 
for  development,  operations,  and  maintenance  should 
be  separate  CIs  with  deliverables  for  users  and  for 
maintainers  clearly  distinguished.  But,  resident 
debug  programs  are  part  of  the  operational  CPC I. 

g.  "Blocks”  are  subdivided  into  "builds"  or  similar 
substructures  and  "builds"  are  the  management 
tracking  items.  The  flow  of  work  and  reviews  pro¬ 
ceed  in  the  usual  fashion. 

h.  A  system  developed  and  managed  by  "builds,"  or  a 
similar  structure,  will  focus  attention  on  user 
capabilities  which  are  demonstrable,  distinguish¬ 
able  and  separate  in  code.  This  will  allow  "accep¬ 
tance"  on  a  block  basis,  with  block  documentation. 

As  a  result,  manpower  peaking  is  reduced,  structure 
in  design  is  emphasized,  and  early  and  orderly  user 
evaluation  and  use  is  permitted. 

i.  The  committee  believes  this  approach  will  reduce 
the  risk  in  development  and  will  yield  lower  devel¬ 
opment  and  life  cycle  costs  and  a  shorter  develop¬ 
ment  period.  In  addition,  surprises  and  scrambling 
will  be  significantly  reduced. 

(AF  Studies  Board,  1 977 » 31 -54) 

In  summary,  the  current  trend  in  software  design  ap¬ 
pears  to  be  to  give  it  better  structure.  This  makes  develop¬ 
ment  of  the  different  parts  by  different  people  much  more 
feasible,  permits  staggered  deliveries,  makes  for  better 
test  and  maintenance,  and  is  a  method  much  to  be  preferred. 

Top-down  design  is  currently  well-publicized  and  has  been 
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effective,  but  is  only  one  of  several  approaches,  no  one  of 
which  seems  to  be  so  superior  that  it  should  become  the  only 
way  ( AP  Studies  Board,  1977  15*0. 


cess.  In  this  unpublished  white  paper,  the  author  reflects 
on  the  lessons  learned  in  developing  systems  at  the  Air 

Force’s  Electronic  Systems  Division(ESD) .  He  proposes  that 
a  fundamental  change  in  attitude  on  the  part  of  management 
at  all  levels  in  regard  to  the  procedures  and  emphasis  for 
the  acquisition  of  these  software  dependent  systems  is  re¬ 
quired.  However,  it  is  not  intended  to  suggest  that  the 
fabric  of  our  acquisition  policies  are  outmoded  and  in  need 
of  replacement,  but  rather  the  need  is  for  a  change  in  the 
way  we  implement  those  policies  (Doane,  1979*1 ). 

As  in  the  previous  study,  the  selection  of  CPCIs  is  not 
directly  addressed  in  this  paper.  However,  the  differences 
between  and  other  weapon  system  acquisitions,  and  a  sam¬ 
pling  of  the  lessons  learned  by  ESD  are  described.  In  ad¬ 
dition,  an  evolutionary  C ^  system  acquisition  strategy  is 
proposed  for  future  acquisitions.  Embodied  in  these  pre¬ 
sentations,  therefore,  are  some  implications  of  considerable 
importance  to  CFCI  selection.  For  example,  the  author 
states,  nCJ  system  requirements  are  intrinsically  evolution-* 
ary,  partly  because  they  must  operate  in  a  constantly,  but 
not  always  predictably,  changing  environment,  and  because 
they  must  support  human  decision  making,  a  process  which 


cannot  be  completely  specified  a  priori.  The  prevailing  Air 
Force  development  and  acquisition  practice  which  treats 
systems  as  if  they  were  stable  rather  than  changing,  as  akin 
to  mass-produced  items  rather  than  one-of-a-kind,  has  been 
out  of  step  with  this  reality.  Thus,  the  author  proposes  to 
subject  C ^  systems  to  an  evolutionary  acquisition  approach 
where  the  system  is  developed  and  delivered  in  multiple 
steps"  (Doane,  1979»9-13).  In  essence,  the  first  step  pro¬ 
vides  some  minimum  capability  and  subsequent  steps  contain 
added  capabilities  as  the  requirements  are  defined,  approved, 
and  budgeted. 

In  other  words,  the  author  believes  that  the  overall 
thrust  of  future  acquisitions  must  be  changed  from  build¬ 
ing  "ideal"  systems  to  building  "workable"  systems.  To  make 
his  point,  he  quotes  Watson  Watt,  who  stated  (in  reference 
to  British  radar  systems),  "...  The  best  design  had  to  be 
rejected  because  it  would  never  be  achieved,  and  that  the 
•second  best’  would  be  achieved  too  late  to  be  used  by  the 
armed  forces  when  they  needed  it.  The  ’third  best'  would  be 
adequate  and  was  available  in  time,  and  it  was  what  won  t  •  ■■> 
Battle  of  Britain. "  He  related  a  more  contemporary  expres¬ 
sion  of  the  same  principle  asi  "If  war  comes,  a  good  system 
in  the  field  is  worth  any  number  of  ideal  systems  on  the 
drawing  board. "  In  short,  we  must  lower  our  sights  and  work 
toward  more  realistic  objectives,  incrementally  building  as 
requirements  evolve  and  are  understood.  That  is,  quantum 
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increases  in  capability  must  be  avoided,  and  incremental 
transfer  of  the  system  to  the  user  must  be  accepted  as  a 
fact  of  life.  System  architectures,  therefore,  must  be 
structured  to  accommodate  change  (Doane,  1979*14-18). 

Joint  Logistics  Commanders  Joint  lolicv  Coordinating 
Group  on  Computer  Resource  -ianarement-Sof tware  Workshop.  A 
panel  on  Software  Acquisition/Development  Standards  was  es¬ 
tablished  during  this  workshop  to  evaluate  the  potential  for 
developing  tri-service  standards  for  the  acquisition  of  ECS 
software.  Currently  each  service  acquires  software  for  ECS 
within  a  very  general  framework  as  typified  by  Military 
Standards  483,  490,  and  1521.  However,  specific  implementa¬ 
tions  vary  widely  between  services  and  even  between  contract¬ 
ing  agencies  within  the  same  service.  Additionally,  experi¬ 
ence  gained  in  the  use  of  the  initial  standards  for  software 
acquisition  has  shown  the  need  for  updating  and  expanding 
them  to  account  for  experience,  earlier  errors  and  omissions, 
and  improved  techniques  of  software  engineering  (Munson^ 
1979«1). 

Once  again,  this  report  did  not  directly  addres^the 
selection  of  CPC  Is.  However,  the  panel  did  look  at  how  each 
service  responded  to  DoDD  5000.29.  evaluated  the  current  Mil 
Standard  structure  to  support  the  policy,  hypothesized  a 
common  software  acquisition  life-cycle  process,  and  where 
possible  identified  areas  where  current  Mil  Standards  are 
inadequate  (Munson,  1979*2).  Thus,  it  is  through  the 
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implementation  of  this  panel's  specific  recommendations  that 
the  previously  cited  conflicts  in  the  DoD  Directives,  AiJPRs, 
and  Mil  Standards  will  be  resolved. 


Reports 

Many  government  and  industry  technical  reports,  masters 
degree  theses,  Air  War  College  publications,  Air  Command  and 
Staff  papers,  and  Defense  System  Management  College  study 
reports  were  reviewed  and  analyzed  in  an  attempt  to  under¬ 
stand  the  various  policies  and  positions  on  software  re¬ 
quirements  allocation  in  the  DoD.  As  in  the  DoD  sponsored 
studies,  there  were  only  a  few  documents  that  directly  ad¬ 
dressed  the  process  of  selecting  CPCIs.  For  the  sake  of  or¬ 
ganization,  they  are  all  summarized  in  this  section  with  one 
exception.  That  is,  the  guidebooks  developed  under  the  Air 
Force  Guidebook  Program  as  a  cooperative  effort  of  the  Air 
Force  and  industry  are  reviewed  in  a  later  section  of  this 
chapter. 

Development  of  Weapon  Systems  Computer  Programs i  A 
Method  for  Control  During  Full-Scale  Development.  Presented 
during  the  Program  Management  Course  at  the  Defense  Systems 
Management  College (DSMC ) ,  this  paper  attempted  to  provide  a 
methodology  and  some  guidelines  which  will  assist  in  con¬ 
trolling  the  development  of  computer  programs  during  full- 
scale  development.  The  author  stresses  the  importance  of 
proper  and  complete  system  requirements  allocation.  He  con¬ 
cluded  that  since  this  is  normally  accomplished  by  the 


contractor’s  systems  engineering  organization  and  the  result 
is  a  subject  for  the  design  reviews,  it  is  imperative  that 
the  government  Program  Manager  have  experienced  computer 
systems  personnel  on  his  staff  (Zabriskie,  1975*20), 

Reliable  Computer  Software!  What  It  Is  and  How  to  Get 
It.  Several  techniques  for  improving  software  design  and 
increasing  software  reliability  are  presented  in  this  DSMC 
paper.  One  of  them,  called  modularization,  is  defined  as 
the  process  of  decomposing  a  program  into  parts  in  order  to 
make  the  system  easier  to  deal  with.  Many  important  bene¬ 
fits  of  proper  modularization  (e.g, ,  manageability)  are  ad¬ 
vocated  by  the  author.  Additionally,  three  criteria  for 
dividing  up  the  effort  involved  in  building  a  large  system 
are  proposed.  They  are« 

a.  tasks  should  be  small  enough  for  one  person  to 
handle , 

b.  decompose  the  system  so  that  management  control  and 
flexibility  can  be  maintained,  and 

c.  partition  the  system  so  that  implementation,  debug¬ 
ging,  and  testing  of  modules  or  groups  of  modules 
can  proceed  in  parallel  (Farnan,  1975*19). 

Configuration  Management  for  the  Development  of  Com¬ 
puter  Systems.  The  author  of  this  report  concluded  that 
configuration  management  is  a  well-defined  discipline  when 
applied  to  classical  major  defense  system  acquisitions  where 
hardware  is  the  prime  item.  That  is,  most  of  the  publica¬ 
tions  within  the  DoD  consider  configuration  management  as  a 
vital  function  and  there  is  considerable  guidance  available 
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for  controlling  hardware  development.  However,  configura¬ 
tion  management  has  had  limited  application  to  embedded  com¬ 
puter  systems  and  even  less  to  general  purpose  computer 
systems.  To  help  alleviate  this  problem,  the  author  pro¬ 
poses  several  actions  for  the  Systems  Program  Office.  For 
example,  CPC  Is  should  be  identified  based  on  whether  they 

a.  satisfy  an  end-use  function, 

b.  are  key  programs, 

c.  have  expected  interface  problems,  and 

d.  will  provide  added  visibility  to  a  particular 
computer  program. 

In  summary  though,  the  author  believes  that  the  selection  of 
a  CPC  I  should  be  based  on  the  best  judgement  of  the  program 
management  team,  and  this  flexibility  will  undoubtedly  re¬ 
sult  in  CPCIs  that  vary  greatly  in  program  size  (Anway, 
1976i13-23). 

Applying  Program  Management  Concepts  to  Software  Devel¬ 


opment!  An  AFR  100-15  Critique.  This  report  proposes 
changes  to  the  recently  published  AFR  300*15  which  adapt  the 
principles  and  concepts  of  program  management  (phases,  base¬ 
lines,  and  formal  reviews)  to  the  software  development  pro¬ 
cess.  Although  both  the  report  and  the  regulation,  entitled 
"Automated  Data  System  Project  Management,"  specifically  ad¬ 
dress  the  acquisition  and  management  of  general  purpose 
systems,  rather  than  embedded  systems,  some  factors  that  af¬ 
fect  the  designation  of  a  CPC  I  are  described  in  the  proposed 

changes.  In  summary,  the  term  CPC  I  is  used  to  describe  the 
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product  or  products  subject  to  the  application  of  configura¬ 
tion  management  procedures.  It  may  be  defined  as  an  entire 
automated  data  system  or  portion  thereof.  That  determina¬ 
tion  is  basically  a  management  decision,  based  on  tradeoffs, 
judgement,  and  experience.  The  factors  bearing  on  the  de¬ 
cision  include  i 

a.  the  risk  associated  with  the  system, 

b.  the  planned  use  of  the  system  (single  or  multiple 
users ) , 

c.  the  complexity  of  the  system, 

d.  the  level  of  change  control  required  by  management, 

e.  the  modification  activity  expected  during  the  life 
of  the  system,  and 

f.  the  cost  thresholds  imposed  by  management  (Caudill, 
1977 : Atch  l,  21). 

System  Acquisition  Guide.  In  May  1978,  this  guide  was 
prepared  as  a  group  research  project  by  students  of  the  Air 
Force  Air  Command  and  Staff  College  at  Maxwell  Air  Force 
Base(AFB),  Alabama.  It  was  not  designed  to  be  a  "how-to-do- 
it"  book*  instead,  it  was  written  principally  as  an  orienta¬ 
tion  for  those  with  very  little  experience  in  system  acquisi¬ 
tion.  Therefore,  information  is  summarized  from  many 
sources,  including  DoD  Directives,  AF  Regulations,  Military 
Standards,  and  manuals  and  guidebooks  from  AF  Systems  Com¬ 
mand  Product  Divisions.  In  the  area  of  CPCI  selection,  very 
little  new  information  was  presented.  In  essence,  the  pro¬ 
cess  was  described  as  requiring  the  exercise  of  judgement 
based  on  experience  and  program  requirements  for  which  no 


set  of  rules  can  adequately  compensate.  In  addition,  two 
considerations  which  can  benefit  the  program  were  presented. 
They  are i 

a.  the  Cl  selection  decision  should  be  made  early  in 
the  program,  and 

b.  care  must  be  taken  to  include  the  Air  Force  Logis¬ 
tics  Command (AFLC )  system  manager  in  this  process 
(Runkle  and  Smith,  1978 t 1 06 ) . 

Model  statement  of  Work (SOW)  Task  for  Software  Develop¬ 
ment.  This  paper  presents  a  model  of  the  general  require¬ 
ments  task  in  a  SOW  used  by  System  Program  Offices (SPO)  at 
the  Air  Force  Electronic  Systems  Division,  Hanscom  AFB,  Mass¬ 
achusetts.  The  model  is  used  as  a  guideline  for  specifying 
the  work  to  be  bid  on  by  prospective  contractors.  In  the 
area  of  software  design,  the  contractor  is  required  to  doc¬ 
ument  the  rationale  supporting  his  system  design  in  accor¬ 
dance  with  the  Contract  Data  Requirements  List(CDRL).  As  a 
part  of  that  design,  a  complete  list  of  CPCIs  must  be  pre¬ 
sented  to  the  Government.  In  addition,  the  rationale  behind 
the  CPC  I  organization  must  be  defined.  To  assist  the  con¬ 
tractor  in  this  task,  four  guidelines  are  provided  by  the 
SPO.  They  are  as  follows i 

a.  The  number  of  CPCIs  shall  be  minimized. 

b.  CPCIs  shall  not  be  organized  across  vendor  product 
lines.  That  is,  a  CPC  I  shall  operate  and  be  tested 
as  a  single  computer  system. 

c.  For  the  purpose  of  this  paragraph,  a  unique  oper¬ 
ating  system (OS)  is  defined  to  be  a  single  vendor 
product  which  controls  the  allocation  of  computer 
resources  for  a  single  vendor  computer  system.  A 
separate  CPC  I  shall  be  provided  for  each  unique  03. 


d.  For  the  purpose  of  this  paragraph,  unique  software 
utility  services  are  defined  as  the  set  of  software 
utility  services  (compilers,  assemblers,  diagnostics, 
and  editors)  which  a  vendor  provides  to  support  a 
single  computer  system.  A  separate  CPC  I  shall  be 
provided  for  each  unique  set  of  software  utility 
services. 

(Model  Statement  of  Work  Task  for  Software  Develop¬ 
ment,  19791^-5) 


Lessons  Learned 

While  software  has  become  increasingly  more  sophisti¬ 
cated  and  complex,  techniques  for  producing  and  ensuring 
quality  in  software  have  not  kept  pace.  As  a  result,  many 
acquisition  programs  encounter  difficult  software  problems. 
Overruns  of  100  percent  in  both  cost  and  the  time  to  develop 
software  have  not  been  unusual  occurrences.  In  addition,  the 
delivered  system  often  fails  to  provide  the  capability  orig¬ 
inally  required.  In  fact,  there  have  been  cases  of  total 
failure  to  develop  systems  (Davis,  1978i19).  Traditionally, 
these  problems  have  been  documented  in  the  form  of  lessons 
learned  by  the  program  office.  Unfortunately,  most  of  them 
do  not  receive  widespread  dissemination.  One  exception  is 
the  set  of  lessons  learned  during  the  acquisition  of  a  large 
command,  control,  and  communication (C^)  system  by  the  Elec¬ 
tronic  Systems  Division  of  AF  Systems  Command. 

In  that  program,  the  magnitude  of  the  problems  spawned 
several  serious  inquiries  to  determine  the  program-specific 
solutions  to  the  problems.  One  of  those  inquiries  concluded 
that  a  configuration  management  system  did  not  exist  on  the 
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program.  More  specifically,  the  functions  of  configuration 

management  existed  in  varying  degrees  and  numerous  forms, 

but  there  was  no  standard.  In  some  cases,  configuration 

management  had  become  so  fragmented  and  dispersed  that  its 

identity  was  almost  lost.  The  lesson  learned  as  a  result  of 

this  problem  was  elegantly  summarized  in  the  October  19?8 

issue  of  "AFSC  Computer  Resource  Newsletter"  by  Mr.  Charles 

Bobbish.  In  supporting  his  conclusions  that  "the  selection 

of  CPCIs  is  critical,"  Mr.  Bobbish  stated, 

"Software  acquisition  should  be  managed 
in  terms  of  system  models  or  versions, 
each  of  which  performs  end-use  system 
functional  capabilities,  and  is  an  evo¬ 
lutionary  outgrowth  of  previous  versions. 

Further,  configuration  management  of 
these  models  should  be  limited  to  base¬ 
lining  and  controlling  (in  the  allocated 
baseline),  the  system  capabilities  con¬ 
tained  in  each  version  as  opposed  to 
baseline  detailed  design  information. 

Why?  If  one  chooses  a  unit  of  manage¬ 
ment  (in  this  case,  a  CPCI),  the  end 
result  of  managing  that  unit  should  be 
the  ability  to  directly  observe  that  a 
meaningful  portion  of  the  system  works. 

A  structure  in  which  the  "functional 
flow"  across  the  system  is  divided  into 
several  CPCIs  is  artificial*  management 
of  a  CPCI  under  this  structure  forces 
the  SPO  to  make  an  inductive  evaluation 
that  the  system  will  ultimately  work  if 
and  only  if  all  other  CPCIs  work" 

(Bobbish,  1978.3). 


Management  Guidebooks 

In  the  past  few  years,  the  discipline  of  software  en¬ 
gineering  has  emerged,  taking  form  as  an  entity  and  formal 
discipline.  The  Air  Force  has  recognized  this  and  initiated 
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efforts  to  provide  guidance  to  its  program  office  personnel. 
The  key  effort  is  the  Software  Acquisition  Management  Guide¬ 
books.  Each  guidebook  takes  a  specific  functional  subject 
(e.g.,  Configuration  Management,  Contracting  for  Software 
Acquisition,  etc.  )  and  emphasizes  practical  approaches  to 
real-world  problems  within  the  context  of  DoD  and  AF  regula¬ 
tions  and  standards.  This  effort  started  in  1975  at  the 
Electronic  Systems  Division  of  AFJC .  Currently  three  sets 
of  guidebooks  are  underway:  C-^,  Airborne  (Avionics,  and 
Space  and  Missile),  and  Ground  Based  (Crew  Trainer/Flight 
Simulator,  and  Automatic  Test  Equipment).  Three  sets  were 
required  to  insure  adequate  coverage  of  the  different  ap¬ 
plication  perspectives  and  to  capture  different  viewpoints 
of  preparation.  Each  set  is  being  prepared  by  a  different 
contractor.  After  completion,  the  guidebooks  will  be  ana¬ 
lyzed  to  see  if  the  texts  can  and  should  be  published  as  one 
integrated  series  (Marciniak,  1978:34-35). 

The  process  of  selecting  CPC  Is  is  addressed  in  varying 
degrees  in  each  of  the  sets.  Basic  principles  governing 
the  selection  in  general  are  presented.  For  example,  les¬ 
sons  learned,  common  pitfalls,  and  mistaken  assumptions  are 
typical  treatments  in  the  guidebooks.  A  comprehensive  sum¬ 
mary  of  each  guidebook  that  addresses  CPC  I  selection  is  not 
provided  in  this  thesis  since  the  information  is  often  over¬ 
lapping  and  furthermore,  this  research  is  primarily  concerned 
with  the  process  of  software  requirements  allocation  to  CPC  Is 


on  a  large  software-intensive  system  that  does  not  inher¬ 
ently  have  a  natural  breakout  (e.g.,  a  system).  However, 
it  is  important  to  emphasize  the  one  common  theme  expressed 
in  several  of  the  guidebooks.  That  is,  the  selection  pro¬ 
cess  is  not  subject  to  "stylized"  rules.  In  the  words  of 
one  C ^  guidebook,  "Decisions  should  be  based  on  experience, 
knowledge  of  the  principles  and  implications,  knowledge  of 
the  given  system  program,  and  attention  to  both  technical 
and  administrative  considerations"  (3earle,  1 977 * 21 ) . 

Software  Acquisition  Management  Guidebook!  Life  Cycle 
Events  (ESD-TR-77-22 ) .  This  guidebook  explains  the  acquisi¬ 
tion  life  cycle  for  major  defense  systems,  the  computer  pro¬ 
gram  life  cycle,  and  their  relationships.  In  the  acquisition 
life  cycle,  the  primary  objective  of  the  second  phase,  vali¬ 
dation,  is  to  assess  the  preferred  design  approach  for  the 
system  (selected  as  a  result  of  the  conceptual  phase)  against 
the  system  requirements.  Once  a  sound  system  design  is 
agreed  to,  the  next  step  in  the  validation  phase  is  to  pro¬ 
vide  clear  technical,  contractual,  economic,  and  organiza¬ 
tional  bases  for  full-scale  development  of  the  system  (Glore, 
1977i21).  Thus,  the  major  product  of  the  validation  phase 
is  the  Allocated  Baseline  (i.e.,  a  set  of  preliminary  devel¬ 
opment  specifications,  one  for  each  CI/CFCI,  which  document 
the  functional,  performance,  interface,  design  and  testing 
requirements  of  the  system  as  they  are  subdivided). 

In  developing  the  allocated  baseline,  the  number  and 


100 


composition  of  CPCIs  is  a  critical  design  issue.  A  system 
of  many  CPCIs  has  many  formally  defined  interfaces.  The 
separate  reports,  specifications,  test  plans/procedures  and 
other  monitoring  activities  required  can  support  good  Gov¬ 
ernment  visibility  into,  and  control  of,  the  development 
process.  However,  if  a  system  is  partitioned  into  too  many 
CPCIs,  the  large  number  of  document  reviews,  engineering 
change  proposals (ECPs ) ,  and  other  management  activities  may 
fragment  insight  and  cause  excessive  delays.  Thus,  the  de¬ 
velopment  of  a  multi-CI  system  may  suffer  more  delay  from 
Government  monitoring  activities  than  a  system  of  fewer 
CPCIs.  Somewhat  different  problems  can  arise  if  the  number 
of  CPCIs  are  few,  but  ill-defined.  For  example,  a  CPC I  may 
contain  processes  that  interact  more  strongly  with  other 
CPCIs  than  with  one  another.  The  inter-CPC  I  interfaces,  al¬ 
though  few,  may  also  be  very  complex.  As  a  result,  the 
larger  scope  of  the  individual  CPC  I  design  reviews  may  fail 
to  identify  many  inconsistencies  among  CPCIs.  These  over¬ 
sights  lead  to  many  ECPs  and  to  progressively  more  expensive 
repairs,  depending  on  when  each  error  is  detected  (Glore, 
1977«26-27). 

Although  the  author  of  this  guidebook  states  that  he 
knows  of  no  well-defined  procedure  for  specifying  an  optimum 
set  of  CPCIs,  he  does  provide  a  list  of  guidelines  that 
should  help  define  a  good  set  of  CPCIs.  In  summary,  they 
are  as  follows » 
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a.  Assign  processes  that  interact  strongly  to  tne  same 
CPCI. 

b.  Assign  processes  that  have  little  or  no  interaction 
to  different  CICIs. 

c.  Allocate  to  different  CPC  Is  processes  that  will 
execute  in  different  computers. 

d.  Assign  to  different  CPC  Is  processes  whose  develop¬ 
ment  can  feasibly  be  finished  at  significantly  dif¬ 
ferent  times,  if  such  phased  development  will  expe¬ 
dite  overall  system  development. 

e.  Allocate  to  different  CPC  Is  software  to  be  procured 
separately. 

f.  Include  in  each  CPCI  no  more  than  a  small,  well- 
knit  group  of  Government  monitors  cam  efficiently 
track,  assuming  reasonable  working  relationships 
between  them  and  the  types  of  personnel  who  will 
manage  and  develop  the  CPCI. 


(Glore,  19?7«27) 

An  Air  Force  Guide  to  Software  Documentation  Require¬ 
ments  (EoD-TR-76-l<S9) .  This  guidebook  identifies  and  de¬ 
scribes  the  most  important  Air  Force  technical  and  manage¬ 
ment  documents  relating  to  software  acquisition.  It  is  in¬ 
tended  for  managers  and  technical  personnel  who  are  respon¬ 
sible  for  determining  the  documentation  requirements  for 
software  in  a  large  system  acquisition.  In  addition,  gen¬ 
eral  conclusions  and  specific  recommendations  for  improving 
software  documentation  are  presented.  One  of  the  conclusions 
specifically  concerns  the  guidelines  for  CPCI  selection.  In 
summary,  the  author  concludes  that,  "Determining  the  number 
of  CPCIs  is  one  management  decision  which  has  a  heavy  impact 
on  software  documentation  requirements"  (Schoeffel,  19?6i 
1>). 
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Since  the  number  of  external  CPC  I  interfaces  increases 
as  more  CPC  Is  are  defined  and  the  documentation  requirement 
is  a  function  of  the  number  of  CPC  Is,  one  approach  to  deter¬ 
mining  the  most  appropriate  number  of  CPCIs  is  to  minimize 
the  CPC  I  interdependencies  and  interfaces.  The  author  rec¬ 
ommends  that  the  number  of  inter-CPC  I  interactions  for  each 
alternative  CPC  I  structure  could  serve  as  a  figure  of  merit, 
along  with  some  other  suitability  factors.  For  example,  the 
other  factors  might  include  adequacy  of  visibility,  degree 
of  development  control  necessary,  degree  of  criticality  or 
risk  involved,  anticipated  contractor  capabilities  (experi¬ 
ence  with  equipment  or  specific  functional  areas),  and  ex¬ 
pected  frequencies  of  changes  in  specific  areas.  In  addition 
to  these  criteria  for  evaluating  the  alternative  CPCI  struc¬ 
tures,  some  of  the  previously  cited  guidelines  for  CPCI  se¬ 
lection  are  reiterated  in  this  guidebook.  To  summarize, 
they  are  as  follows i 

a.  Define  separate  CPCIs  for  capabilities  developed  or 
delivered  at  different  times. 

b.  Separate  operational  programs  from  utility  programs 
and  those  from  diagnostic  programs, 

c.  Separate  out  particular  programs  with  potential  use 
in  multiple  systems. 

d.  Combine  highly  interrelated  computer  programs  into 
a  single  CPCI. 


(Schoeffel,  1 976 « 134-1 35) 

An  Air  Force  Guide  to  Computer  Program  Configuration 
Management  (ESD-TR-77-254).  Of  all  the  sources  reviewed  in 
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this  extensive  literature  search,  this  document  provides  the 
most  comprehensive  treatment  of  the  CICI  selection  process. 

It  is  written  as  a  combined  instructional  and  reference  doc¬ 
ument  which  identifies,  interrelates,  and  explains  the  re¬ 
quirements  of  current  Air  Force  and  DoD  configuration  manage¬ 
ment  standards.  Emphasis  is  placed  on  the  acquisition  of 
computer  resources  in  major  defense  system  programs.  Spe¬ 
cific  background  information,  guidance,  pitfalls,  and  ex¬ 
amples  are  devoted  to  software  elements  of  C-^  programs 
(Searle ,  1977*1). 

This  guidebook  defines  the  selection  of  configuration 
items  as,  "The  process  by  which  the  complete  set  of  equip¬ 
ment,  computer  programs,  and  facilities  elements  contem¬ 
plated  for  a  system  as  a  whole  are  separated  for  purposes 
of  managing  their  development  or  other  procurement  into  in¬ 
dividually-identified  subsets."  Hence,  the  configuration 
item  is  regarded  as  a  level  of  management.  Specifically,  it 
is  the  level « 

a.  at  which  the  procuring  activity  specifies,  con¬ 
tracts  for,  and  accepts  individual  parts  of  the 
system, 

b.  below  which  the  developer  is  responsible  for  man¬ 
agement  of  the  development,  or  procurement,  and  as¬ 
sembly  of  item  components,  and 

c.  above  which  the  procuring  activity  retains  respon¬ 
sibility  for  interfaces,  integration,  and  system 
performance  (Searle,  1977 * 21 ) . 

Thus,  the  CPC  I  is  the  level  at  which  a  program  office  ex¬ 
ercises  formal  management  control  over  the  responsible 
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contractor  in  the  areas  of  configuration  management,  pro¬ 
curement,  program  control,  and  technical  progress. 

The  initial  selection  of  CPCIs  is  usually  accomplished 
at  an  early  stage  of  the  acquisition  program.  It  represents 
a  design  decision  and  is  basically  a  technical  product  of 
the  system  engineering  process.  In  summary,  the  identifica¬ 
tion  of  a  given  assembly  of  computer  program  instructions 
and  coded  data  as  a  CPC  I  results  from  the  steps  ofi 

a.  functional  analysis  and  definition  of  system  per¬ 
formance  requirements,  and 

b.  system  design,  during  which  the  defined  functional 
and  performance  requirements  are  allocated  among 
planned  assemblies  of  system  physical  elements. 

Therefore,  the  CFC I  designation  constitutes  a  commitment  to 

develop  a  deliverable  end  item  (e.g,,  in  the  form  of  a  tape 

or  deck  of  cards)  which  will  perform  its  allocated  functions 

when  eventually  assembled  into  the  system  (Searle,  1977 «21- 

22). 

Like  several  other  studies  and  technical  reports,  this 
guidebook  provides  a  "shopping  list"  of  criteria  to  be  used 
in  selecting  CPCIs.  First  of  all,  the  author  proposes  that 
the  intended  source  (contractor)  is  an  essential  starting 
point  for  decisions,  since  assemblies  of  computer  program 
elements  to  be  acquired  from  a  single  contractor  are  poten¬ 
tially  a  single  CPCI,  and  assemblies  to  be  acquired  from 
separate  sources  must  be  separate  CPCIs.  Factors  of  cost, 
complexity  of  documentation,  interface  control,  and  other 
requirements  dictate  that  it  is  generally  desirable  to  avoid 


having  any  more  CIO  Is  than  necessary.  Therefore,  for  a  giv¬ 
en  single  contractor,  a  productive  approach  is  to  start  with 
the  tentative  assumption  of  a  single  CPCI,  then  "shredout" 
into  separate  CPC  Is  only  when  fully  justified  by  an  appli¬ 
cable  criterion  such  as  one  of  the  following! 

a.  Separate  Computers.  Computer  programs  to  be  de¬ 
signed  for  operation  in  different  types  or  models 
of  computers  must  be  separate  CPCIs. 

b.  Separate  Schedules.  Computer  programs  scheduled 
for  development,  testing,  or  delivery  at  different 
times  may  be  separate  CICIs. 

c.  Different  System  Functions  and  Uses.  In  general, 
mission,  support,  and  diagnostic  (off  line)  computer 
programs  should  be  separate  CPCIs. 

d.  Different  Deployment  Phase  Control.  Computer  pro¬ 
grams  intended  for  different  systems,  and/or  for 
different  conf iguration  control  during  the  deploy¬ 
ment  phase,  should  be  identified  as  separate  CICIs, 
even  though  they  may  be  largely  identical  at  the 
time  of  initial  deployment  and  delivery. 

(Searle ,  1977i23-24) 

In  summary,  the  author  stated,  "Although  a  single 
'right'  solution  may  not  always  present  itself,  reasonable 
care  and  attention  to  the  considerations  outlined  above 
should  yield  sound  results.  On  the  other  hand,  success  of 
the  program  can  be  almost  precluded  if  those  objectives  and 
principles  are  disregarded"  (Searle,  1977 « 24). 

Software  Acquisition  Management  Guidebook!  Reviews  and 
Audits  (h'SD-TK-78-117).  This  report  combines  existing  guid¬ 
ance  regarding  reviews  and  audits  currently  contained  in  a 
number  of  official  documents  into  a  single  document,  and 
narrows  the  focus  of  existing  guidance  to  those  problems 
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associated  with  software  acquisition  management.  Detailed 
guidance  is  provided  for  the  engineering  design  reviews 
which  are  conducted  on  the  system  during  the  validation 
phase  to  evaluate  the  CI/CFCI  definitions  and  to  determine 
if  system  design  compatibility  has  been  maintained  (Neil, 
1977  *  5—1 1 ). 

During  the  early  stages  of  the  validation  phase,  a 
System  Requirements  Review  is  conducted  to  monitor  the  con¬ 
tractor's  system  engineering  activities.  For  example,  the 
refined  system  requirements  and  preliminary  C I/CPC  I  selec¬ 
tions  are  reviewed.  These  preliminary  decisions  normally 
result  from  initial  trade-off  studies  conducted  by  the  con¬ 
tractor.  Although  the  initial  CPC  I  breakout  is  done  on  a 
technical  basis,  the  final  decision  must  be  tempered  by  man¬ 
agement  and  acquisition  issues.  In  fact,  the  author  of  this 
guidebook  asserted i 

"Many  acquisition  problems  are  created 
if  CPCI  definitions  are  only  viewed 
technically.  A  common  misconception 
is  that  more  CIs  will  provide  more  con¬ 
trol  and  visibility.  In  fact,  when  many 
CPC  Is  are  defined,  costs  are  increased 
(increased  data,  management  reviews,  and 
control,  and  configuration  management 
requirements),  the  program  office  accepts 
more  responsibility  (an  increase  in  the 
number  of  development  specifications, 
each  with  interfaces  that  must  be  ap¬ 
proved  by  the  Government  when  the  speci¬ 
fication  is  baselined),  and  the  visibil¬ 
ity  remains  essentially  the  same  (the 
development  specification  requirements 
must  still  be  defined  at  the  performance 
level)"  (Neil,  1977«19). 
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Conference  Proceedings 

In  the  past  few  years,  papers  addressing  *he  acquisi¬ 
tion  and  management  of  embedded  computer  resources  have  been 
presented  at  numerous  conferences  and  symposiums.  Unfortu¬ 
nately,  most  of  them  are  primarily  concerned  with  the  tech¬ 
nical  aspects  of  software  engineering  and  programming  meth¬ 
odologies  (e.g.,  requirements  analysis,  top-down  versus  bot¬ 
tom-up  design,  etc.).  In  other  words,  the  process  of  al¬ 
locating  software  requirements  to  CPC  Is  was  rarely  addressed 
although  CPCI  selection  is  a  technical  decision  tempered 
with  management  considerations.  This  fact  is  somewhat  dis¬ 
turbing  considering  the  effect  this  decision  has  had  on  the 
overall  success  of  many  large  weapon  system  acquisition  pro¬ 
grams.  However,  a  few  authors  have  discussed  the  need  for 
better  techniques  for  decomposing  software  requirements  in 
general.  For  example,  the  presenter  of  one  paper  at  the 
Second  International  Conference  on  Software  Engineering  con¬ 
cluded,  "A  technology  needs  to  be  developed  that  permits  the 
structured  decomposition  of  the  system  specification  into  a 
complete  and  consistent  set  of  data  processing  subsystem  re¬ 
quirements"  (Belford,  e£  ai.. ,  19?6i72).  This  paper  and  the 
others  relevant  to  this  research  are  summarized  in  the  fol¬ 
lowing  paragraphs. 

Specif lcations >  A  Key  to  Effective  Software  Develop¬ 
ment.  In  supporting  the  above  conclusion,  the  author  of  this 
paper  states,  "The  initial  steps  in  the  subsystem  engineering 
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phase  should  include  the  decomposition  of  the  (system)  spec¬ 
ification  into  discrete  processes  and  the  verification  of 
these  processes.  Therefore,  this  decomposition  process 
should  trace  the  original  system  requirements  specifications 
from  one  level  of  decomposition  to  another  (vertical  trace- 
ability)  and  also  trace  each  of  these  requirements  within 
any  given  level  of  decomposition  (horizontal  traceability)" 
(Belford,  et  a^, ,  1976i72).  This  would  hopefully  provide 
the  capability  to  identify  inherent  conflicts,  discrepancies, 
and  omissions  and  ensure  the  completeness  of  each  require¬ 
ment  defined  in  the  system  specification.  In  summary  though, 
the  author  admits,  "The  key  issue  remaining  in  the  decompo¬ 
sition  technology  is  performance  allocation"  (Belford,  ejt 
al. ,  1976*79). 

Managing  the  Develonment  of  Reliable  Software.  This 
paper,  presented  at  the  1975  International  Conference  on 
Reliable  Software,  highlights  a  number  of  techniques  and 
project  activities  which  have  proven  to  be  especially  valu¬ 
able  to  the  TRW  Defense  and  Space  Systems  Group  in  their 
efforts  to  develop  reliable  software.  In  analyzing  the 
"specif ics-what  have  we  done  and  what  have  we  learned,"  the 
author  related  the  following* 

a.  There  is  nothing,  absolutely  nothing,  that  can 

cause  a  more  devastating  outcome  of  a  development 
activity  than  an  incomplete  and/or  incorrect  know¬ 
ledge  and  corresponding  specification  of  system  and 
software  requirements. 
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b.  No  matter  how  you  slice  it,  the  creation  of  a  high¬ 
ly  sophisticated,  yet  inplementaole  design  solution 
which  will  clearly  satisfy  demanding  design,  func¬ 
tional  processing  and  performance  requirements  is 
far  from  an  easy  task, 

c.  We  need  to  reduce  the  normal  wait  time  before  one 
could  "hear  the  process  talk. "  An  incremental  ap¬ 
proach  to  detailed  design,  code,  and  test  in  con¬ 
junction  with  to]r-down  concepts  is  possible  to  im¬ 
plement  through  dummy  processes,  loops,  end  builds. 
In  other  words,  within  that  structure,  it  is  pos¬ 
sible  to  identify  selected  portions  of  the  software 
(e.g.,  processing  threads  involving  critical  func¬ 
tions)  which  can  be  developed  and  tested  as  inde¬ 
pendent  increments.  With  care,  these  increments 
(or  loops)  can  be  chosen  to  provide  a  complete  in¬ 
crement  of  system  capability  and,  thereby,  substan¬ 
tially  reduce  the  likelihood  of  last-minute  sur¬ 
prises. 


(Williams,  1975*2-4) 
Reauiroments-Oriented  Design t  An  emerging  Design  stra¬ 
tegy.  In  this  paper  presented  at  the  Spring  1977  COMPCOM, 
the  author  proposes  that  a  user’s  needs  may  be  broken  into 
two  categories*  Requirements  and  Attributes.  Basically, 
requirements  are  the  constraints  which  the  system  must  sat¬ 
isfy  (i.e,,  what  the  system  "must  do").  Attributes,  on  the 
other  hand,  specify  either  options  or  evaluation  criteria 
for  qualitative  comparisons  of  competing  systems  that  meet 
the  system  requirements.  For  example,  attributes  may  be 
used  to  evaluate  competing  architectures  to  obtain  a  feel 
for  the  "goodness"  of  the  architecture  in  solving  ' .<«  cus¬ 
tomer’s  problem.  Although  some  of  them  do  not  apply  to 
software,  several  criteria  are  presented  for  use  in  deter¬ 
mining  the  required  attributes.  They  are*  flexibility,  ex¬ 
pandability,  bus  complexity,  executive  complexity, 
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availability,  partitioning,  modularity,  reliability,  main¬ 
tainability,  manufacturability,  production  cost,  development 
cost,  technical  risk,  logistics,  programmability,  support 
software  cost,  software  adaptability  and  transferability, 
compatability,  and  service  (Thurber,  1977 « 305-307) . 

Periodicals 

Like  the  papers  presented  at  various  conferences  and 
symposiums,  articles  in  the  numerous  management  and  techni¬ 
cal  periodicals  have  seldom  addressed  the  criteria  for  se¬ 
lecting  CPCIs  and  the  impacts  of  that  decision  on  other  as¬ 
pects  of  the  system  program.  However,  incremental  develop¬ 
ment  has  been  the  subject  of  a  few  articles.  Similarly, 
some  research  has  been  performed  on  the  criteria  to  be  used 
in  decomposing  system  requirements  into  modules.  For  exam¬ 
ple,  D.L.  Parnas  proposed  "information  hiding"  as  a  technique 
for  decomposing  systems  into  modules  (Parnas,  19?2«1053). 
Since  the  fifth  objective  of  this  research  is  to  determine 
the  feasibility  and  evaluate  the  potential  impacts  of  de¬ 
fining  CPCIs  in  terms  of  system  versions  or  models  (to  be 
developed  incrementally),  the  more  relevant  points  in  these 
articles  are  summarized  below. 

Perspectives  on  Software  Engineering.  As  indicated  by 
Marvin  V.  Zelkowitz  in  this  article,  each  stage  of  the  soft¬ 
ware  development  life  cycle  has  its  own  set  of  problems  and 
solutions.  The  most  advanced  techniques  apply  to  the  latter 
stages  since  the  first  stages  are  the  least  developed.  For 
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example,  testing  and  debugring  tools  are  apparent  to  every 
programmer  since  they  are  the  oldest  and  most  advanced.  On 
the  other  hand,  optimum  design/development  strategies  are 
not  readily  available  for  each  situation.  However,  the  au¬ 
thor  doe3  describe  one  approach  which  gives  the  user  a  run¬ 
ning  system  early  in  the  life  cycle  when  changes  are  easier 
to  make.  This  approach,  called  iterative  enhancement  (or 
incremental  development),  is  another  technique  (like  top- 
down  development)  for  implementing  hierarchically  structured 
programs.  In  this  technique,  a  subset  of  the  problem  is 
first  designed  and  implemented.  Then,  the  process  is  re¬ 
peated  to  develop  successively  larger  subsets  until  the  fi¬ 
nal  product  is  delivered.  At  each  step  of  the  process,  not 
only  extensions  but  also  design  modifications  can  be  made. 
However  fewer  and  fewer  modifications  are  needed  as  the  it¬ 
erations  converge  to  the  full  system.  Thus,  iterative  en¬ 
hancement  can  make  rebuilding  less  chaotic  since  there  is  a 
running  system  early  in  the  development  cycle  (Zelkowitz, 
1978.207-212). 

Iterative  Enhancement »  A  Practical  Technique  for  .Soft¬ 
ware  Development.  The  authors  of  this  article,  Victor  R. 
Basili  and  Albert  J,  Turner,  are  generally  credited  with 
demonstrating  this  technique  as  a  practical  means  of  using  a 
top-down,  stepwise  refinement  approach  to  software  develop¬ 
ment.  In  summary,  their  article,  published  in  the  IEEE 
Transactions  on  Software  Engineering,  in  December  1975. 


states  < 


The  development  and  quantitative  anal¬ 
ysis  of  a  production  compiler  for  the 
language  SIM1L-T  was  used  to  demonstrate 
that  the  application  of  iterative  en¬ 
hancement  to  software  development  is 
practical  and  efficient,  encourages  the 
generation  of  an  easily  modifiable  pro¬ 
duct,  and  facilitates  reliability 
(Basil!  and  Turner,  1975»390). 

However,  the  need  for  experimental,  iterative  design  was 

recognized  as  early  as  1973  when  the  CC IP-85  Study  drew  the 

following  conclusion  in  Volume  VII,  entitled  "Technology 

Trends i  Integrated  Design"  of  their  report. 

One  of  the  most  recurrent  themes  of  this 
volume  is  the  notion  using  "experimental 
design"  to  determine  explicitly,  and  on 
the  most  cost-effective  basis,  the  opti¬ 
mum  configuration  and  capabilities  for  a 
C&C  system.  Perhaps  "experimental  de¬ 
sign"  is  an  inaccurate  description;  "de¬ 
sign  through  testing,  modifications,  and 
validation  of  increments  of  a  system  for 
maximum  effectiveness"  may  be  more  apt 
(Boehm,  1972«44). 


Textbooks.  Essays,  and  Academic  Facers 

Many  authors  have  contributed  to  our  understanding  of 
the  problems  associated  with  software  development  by  pre¬ 
senting  the  theoretical  aspects  in  textbooks,  essays,  and 
academic  papers.  All  phases  of  the  software  development 
life  cycle  have  been  discussed  in  this  academic  arena.  How¬ 
ever,  most  of  the  emphasis  has  been  on  requirements  analysis 
and  system  design.  For  example,  automated  techniques  for 
analyzing  system  requirements,  methods  for  decomposing  the 
requirements  into  modules  and  subroutines,  and  design 
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considerations  such  as  cohesion  and  coupling  have  all  re¬ 
ceived  a  lot  of  attention.  As  a  result,  only  one  publica¬ 
tion  relevant  to  the  process  of  selecting  CPC  Is  for  the  pur¬ 
pose  of  managing  their  development  is  referenced  in  this 
section. 

Student  Workbook  for  Course  B»  Software  Acquisition 
and  Management.  This  document  was  published  by  System  De¬ 
velopment  Corporation  and  used  as  a  handout  in  tneir  Avion¬ 
ics  Software  Training  Program.  In  the  configuration  manage¬ 
ment  and  identification  section  the  following  six  criteria 
for  CPCI  selection  are  presented. 

a.  CPC  Is  are  not  part  of  an  equipment  Cl. 

b.  Number  of  CPCIs  and  interfaces  should  be  minimized. 

c.  Size  is  not  a  criterion. 

d.  Separate  CPCIs  should  be  defined  for  separate  con¬ 
tractors  , 

e.  Separate  CPCIs  should  be  defined  for  separate  users/ 
managers. 

f.  CPCIs  are  the  basis  for  support  documents. 

(Searle,  197^*25) 


Chapter  Summary 

As  more  and  more  of  the  processes  which  control  our 
life-styles  are  automated,  a  higher  level  of  trust  is  placed 
in  the  reliable  functioning  of  this  proliferating  technology. 
During  the  past  10  years,  the  problems  associated  with  auto¬ 
mating  these  processes  have  been  extensively  studied  by  tech¬ 
nical  and  management  analysis  teams.  Thus,  there  is  an 
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abundance  of  information  published  concerning  the  acquisi¬ 
tion  and  management  of  computer  resources.  This  information 
emanated  from  various  sources,  and  in  most  cases  was  di¬ 
rected  at  specific  functional  areas.  Since  the  documents 
reviewed  and  summarized  in  this  section  were  of  the  non- 
regulatory  type,  studies,  technical  reports,  conference  pro¬ 
ceedings,  textbooks,  essays,  lessons  learned,  periodicals, 
and  guidebooks  were  included. 

The  process  of  allocating  system  software  requirements 
to  CPCIs  was  not  the  specific  subject  of  any  published  in¬ 
formation  discovered  in  this  research.  However,  several 
criteria  and  factors  to  be  considered  in  the  selection  of 
CPCIs  were  presented  in  several  publications.  The  more  im¬ 
portant  ones  are  as  follows « 

a.  Premature  ’’freezing"  of  system  structure  into  man¬ 
agement  structure  causes  problems (Ref .  Para,  a,  p.  81). 

b.  Each  operational  program  should  be  a  CPC I. 

c.  Software  tools  should  be  separate  CPC  Is ;  however 
resident  debug  programs  are  part  of  the  operation 
CPCI. 

d.  CPCIs  should  be  small  enough  for  one  person  to 
manage . 

e.  Decompose  the  system  so  that  management  control  and 
flexibility  can  be  maintained. 

f.  Partition  the  system  so  that  implementation,  de¬ 
bugging,  and  testing  of  modules  or  groups  of  mod¬ 
ules  (CPCIs)  can  proceed  in  parallel. 

g.  CPCIs  should  be  identified  based  on  whether  they 
satisfy  an  end-use  function,  are  key  programs,  have 
expected  interface  problems,  and  will  provide  added 
visibility  to  a  particular  computer  program. 
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h.  The  CPC  I  selection  decision  should  be  made  early  in 
the  program. 

i.  Care  must  be  taken  to  include  the  AFLC  system  man¬ 
ager  in  the  selection  process. 

j.  The  number  of  CPC  Is  should  be  minimized. 

k.  A  CPCI  must  be  operated  and  tested  as  a  single  com¬ 
puter  system. 

l.  The  end  result  of  choosing  a  CPCI  as  a  unit  of  man¬ 
agement  should  be  the  ability  to  directly  observe 
that  a  meaningful  portion  of  the  system  works. 

Thus,  a  structure  in  which  the  "functional  flow" 
across  the  system  is  divided  into  several  CPCIs  is 
artificial. 

Several  other  management  considerations  were  included 
in  this  chapter.  For  example,  the  risk,  planned  use,  com¬ 
plexity,  level  of  change  control  required,  expected  modifi¬ 
cation  activity,  and  cost  thresholds  were  all  identified  as 
factors  bearing  on  the  CPCI  selection  decision.  In  addition, 
the  findings  and  recommendations  of  the  Air  Force  Studies 
Board,  Committee  on  Operational  Software  Management  and  De¬ 
velopment,  regarding  the  incremental  development  approach 
were  summarized.  Although  this  appears  to  be  unrelated  to 
CPCI  selection  criteria,  it  was  included  in  this  literature 
search  since  the  fifth  objective  of  this  research  was  to 
determine  the  feasibility  and  evaluate  the  potential  impacts 
of  defining  CPCIs  in  terms  of  system  versions  or  models. 


V. 


This  chapter  presents  and  discusses  the  results  of  the 
analysis  performed  on  the  interview  data  collected  in  this 
research  effort.  The  remainder  of  this  chapter  is  devoted 
to  presenting  demographics  of  the  sample  population,  char¬ 
acteristics  of  the  development/acquisition  program  evalu¬ 
ated,  CPC  I  selection  criteria  on  that  program,  and  the  re¬ 
spondents’  opinions  on  the  feasibility,  potential  impacts, 
and  implementation  of  horizontal  allocation  as  a  method¬ 
ology  for  allocating  software  requirements  to  CPCIs. 


Description  of  the  Sample  Population 

The  sample  population,  as  described  in  Chapter  II,  in¬ 
cluded  45  respondents  representing  10  Air  Force  organiza¬ 
tions,  2  Federal  Contract  Research  Centers,  and  11  contrac¬ 
tors  of  the  U.S.  Aerospace  industry.  Although  the  limited 
time  available  for  the  research  did  not  permit  interviews 
with  a  larger  sample,  the  various  backgrounds  and  areas  of 
expertise  reported  by  the  respondents  established  a  data 
base  of  facts  and  expert  opinions  that  was  assumed  to  be 
representative  of  the  overall  population.  In  addition,  the 
sample  was  assumed  to  be  sufficiently  large  to  support  the 
overall  conclusions  and  recommendations  made  in  Chapter  VI. 
These  assumptions  appear  to  be  reasonable  in  light  of  the 
descriptive  statistics  for  the  sample  demographics. 
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Descriptive  Statistics.  Information  on  the  distribu¬ 
tion,  variability,  and  central  tendencies  of  each  demograph¬ 
ic  is  shown  in  Appendix  G,  Table  VI.  These  descriptive 
statistics  reflect  a  sample  population  that  was  highly  ed¬ 
ucated,  trained,  and  experienced.  For  example,  15  different 
software  acquisition  and  management  disciplines  were  rep¬ 
resented  in  the  sample.  Likewise,  the  grade  level  for  the 
respondents  ranged  from  Captain  to  General  for  military  per¬ 
sonnel,  GS-12  to  G3-16  for  civil  service  personnel,  and  pro¬ 
grammer/analyst  to  vice-president  for  contractor  personnel. 
Although  only  about  half  of  the  sample  have  had  any  formal 
training  in  either  system  engineering,  software  engineering, 
hardware  engineering,  or  acquisition  management,  more  than 
half  of  the  respondents  reported  their  educational  level  as 
completion  of  a  masters  degree. 

The  respondents  were  practically  all  (91.1#)  currently 
working  in  the  development/acquisition  of  computer  software. 
Furthermore,  the  average  respondent  had  been  involved  with 
these  types  of  programs  for  more  than  12  years.  This  ex¬ 
perience  was  spread  over  more  than  6  system  programs.  A 
large  majority  of  the  respondents  (82.2#)  indicated  the  ac¬ 
quisition  and  management  of  computer  software  embedded  in 
systems  was  the  primary  emphasis  of  their  experience.  About 
one-fourth  of  the  respondents  reported  having  experience  in 
the  other  major  categories  of  software  application.  This 
diversity  of  system  application  experience  was  repeated  in 
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the  type  of  software  experience  the  sample  population  had. 
For  example,  more  than  half  of  those  interviewed  indicated 
they  had  experience  in  pi ogramming,  systems  analysis,  and/or 
test  and  evaluation.  Similarly,  more  than  one-third  of  the 
sample  population  were  experienced  in  software  maintenance, 
system  design,  and/or  acquisition  management. 

Analysis  of  Variable  Interdependency .  Some  bivariate 
correlations  among  the  demographic  variables  were  statisti¬ 
cally  significant  (Ref.  Appendix  J,  Table  IX),  However, 
of  them  were  unexpected.  For  example,  the  number  of  system 
acquisition  programs  a  respondent  had  worked  on  was  corre¬ 
lated  with  the  number  of  years  of  experience  he  had  in  the 
development/acquisition  of  software. 

Program  Characteristics 

In  order  to  evaluate  the  criteria  presently  being  used 
by  SPOs  and  contractors  for  allocating  system  requirements 
to  CPCIs,  the  respondents  were  asked  to  identify  a  specific 
program,  and  then  to  answer  a  series  of  questions  relating 
to  those  criteria.  These  questions  included  attributes  of 
the  program,  identification  of  the  selection  criteria,  ad¬ 
vantages/disadvantages  of  the  criteria,  and  problems  with 
the  CPC  I  structure  of  the  program.  Some  of  these  questions 
asked  for  specific  values,  such  as  the  number  of  CPCIs  on 
the  program.  Others  requested  the  respondent  to  provide  a 
yes/no  answer  and  then  briefly  expand  on  his  answer.  Fi¬ 
nally,  some  questions  asked  the  respondent  to  provide  a 
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subjective  answer  (e.g.,  what  changes,  if  any,  ne  would  like 
to  see  in  the  way  CPC  Is  are  selected).  The  statistical  anal¬ 
ysis  results  for  each  program  characteristic  question  that 
is  quantifiable  are  presented  in  Appendices  H  (descriptive 
statistics)  and  J  (statistically  significant  bivariate  cor¬ 
relations).  In  addition  to  these  statistical  results,  the 
qualitative  responses  are  summarized  in  the  following 
sections. 

Program  Identification?  Although  the  scope  of  this  re¬ 
search  was  limited  to  large  software-intensive  systems,  40 
different  system  development/acquisition  programs  were  iden¬ 
tified  and  discussed  by  the  respondents.  A  listing  of  these 
is  attached  as  Appendix  K. 

Years  Assigned  to  the  irogram  (328)?  The  length  of  time 
the  respondents  worked  on  their  identified  program  ranged 
from  less  than  1  year  to  over  10  years,  with  an  average  of 
3^  years. 

Respondent's  Pcb  on  the  Program?  All  activities  in¬ 
volved  in  the  system  acquisition  and  software  life  cycles 
were  represented  in  the  job  descriptions  of  those  personnel 
interviewed. 

When  Were  System  Requirements  Allocated  to  CPCIs? 

System  requirements  were  allocated  to  CPCIs  in  most  cases  at 
the  outset  •>!  each  contract.  Of  the  40  programs  reported 
on,  CPCI  selection  was  done  13  times  during  the  conceptual 
phase  and  l6  times  during  the  full-scale  development  phase. 


The  other  answers  were  about  evenly  spread  among  validation 
phase,  unknown,  and  not  done  yet.  However,  on  one  program 
the  seloction  wa3  performed  as  late  as  the  critical  design 
review (CDR).  Furthermore,  one  respondent  indicated  that  no 
allocation  was  accomplished  until  the  contractor  got  in 
trouble  during  development. 

Who  Performed  CPC  I  Selection?  The  initial  allocation 
of  requirements  to  CPC  Is  was  performed  by  the  contractor  for 
more  than  half  of  the  programs.  In  most  of  those  cases,  the 
contractor  proposed  the  CPC  I  structure  and  the  Government 
modif ied/approved  it.  However,  Government  and/or  Federal 
Contract  Research  Center  personnel  did  allocate  the  system 
level  requirements  to  CPC  Is  on  13  of  the  ^0  programs.  In 
these  cases,  the  CPC  Is  were  defined  either  in  the  system 
level  specification  or  statement  of  work,  and  then  directed 
upon  the  contractor.  Finally,  only  one  respondent  indicated 
that  he  was  unaware  of  who  selected  the  CPCIs  on  his  program. 

Program  Development  Cost  (Q2S)?  The  programs  repre¬ 
sented  in  this  reseaxch  ranged  from  less  than  $1  million  to 
$500  million  in  total  development  cost.  Although  more  than 
2rjfo  of  the  respondents  indicated  that  cost  data  w^a  unknown 
or  not  available  to  them,  a  sufficiently  large  sample  was 
available  for  application  of  statistical  techniques.  This 
analysis  provided  two  interesting  results.  First,  a  large 
number  of  the  programs  were  in  the  $100  million  to  $500  mil¬ 
lion  category.  Secondly,  simple  bivariate  correlations 
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between  the  program  characteristic  variables  and  the  demo¬ 
graphic  variables  indicate  a  positive  correlation  coeffi¬ 
cient  of  0.40  between  experience  in  software  engineering  and 
total  development  cost.  In  other  words,  personnel  assigned 
to  the  large  programs  have  more  software  engineering  ex¬ 
perience  than  those  assigned  to  the  smaller  programs. 

Number  of  CPCIs  (Q10)?  A  large  amount  of  variance  was 
reflected  in  the  answers  to  this  question.  In  other  words, 
the  number  ranged  from  1  to  200,  with  an  average  of  21  and 
a  standard  deviation  of  27.069.  In  some  cases,  the  number 
of  CPCIs  delivered  to  the  user  was  considerably  smaller  than 
what  was  initially  selected.  One  respondent  explained  this 
situation  by  stating,  "Once  the  contractor  got  into  trouble 
and  development/integration  couldn’t  proceed,  the  CPC  I 
structure,  which  was  forced  on  the  contractor  in  a  bizarre 
way,  had  to  be  redefined  because  functions  were  cutting 
across  lots  of  CPCIs  in  an  illogical  fashion." 

The  simple  bivariate  correlations  performed  on  the  de¬ 
mographic  and  program  characteristic  variables  revealed  two 
interesting  results.  First,  the  number  of  CPCIs  and  a  re¬ 
spondent’s  opinion  as  to  the  adequacy  of  the  analysis  per¬ 
formed  on  the  alternative  CPC  I  structures  were  negatively 
correlated  with  a  correlation  value  of  -0.37.  This  result 
suggests  that  an  adequate  analysis  of  the  alternative  CPC  I 
structures  might  reduce  the  number  of  CPCIs  on  a  software¬ 
intensive  system.  Secondly,  there  is  a  statistically 
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significant  positive  correlation  between  the  number  of  CPCIs 
and  whether  the  respondent  had  any  formal  training  in  system 
engineering,  software  engineering,  hardware  engineering,  and 
acquisition  management.  Therefore,  more  trained  personnel 
were  assigned  to  larger  programs  (in  terms  of  the  number  of 
CPCIs)  than  to  the  smaller  ones. 

Number  of  Lines  of  Code  (Q31)?  Over  80%  of  the  programs/ 
projects  represented  in  the  interviews  contained  computer 
software  that  exceeded  100,000  lines  of  code  (i.e.,  execu¬ 
table  instructions).  This  result  was  expected  since  the  ini¬ 
tial  plan  was  to  only  interview  people  with  experience  on 
software-intensive  systems.  In  addition,  the  number  of  lines 
of  code  was  significantly  correlated  with  total  development 
cost  (0.75),  number  of  CPCIs  (0.62),  and  the  respondent's 
opinion  as  to  the  adequacy  of  the  analysis  performed  on  the 
alternative  Cf'CI  structures  (0.43). 

Cost  Ratio-Software  to  Hardware  (Q32)?  Only  4y>  of 
those  interviewed  answered  this  question »  and  several  of 
those  that  did  indicated  their  answer  was  strictly  an  esti¬ 
mate.  Therefore,  the  validity  of  any  conclusions  based  on 
the  responses  to  this  question  is  questionable. 

Percent  Contract  Complete?  The  percentage  of  the  soft¬ 
ware  development/contract  period  that  was  complete  ranged 
from  2%  to  100%,  with  an  average  of  69%.  In  other  words, 
all  stages  of  the  software  development  cycle,  including  100% 
complete,  were  represented  in  the  reported  programs. 
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.?  A  majority  (62%)  of 


CPC  Is  Defined  at  Right  Size  (QVO 
the  respondents  answering  this  question  believed  that  the 
CP'CIs  were  defined  at  about  the  right  size  in  terms  of  man¬ 
aging  their  development.  This  result  was  negatively  corre¬ 
lated  with  the  cost  ratio  of  software  to  hardware  at  the 
O.53  level.  In  other  words,  as  the  cost  ratio  of  software 
to  hardware  increased,  the  respondents  tended  to  believe 
that  the  CPC  Is  were  not  defined  at  about  the  right  size.  In 
addition,  some  of  the  problems  that  can  occur  when  CPC  Is  are 
improperly  sized  were  described  by  those  responding  nega¬ 
tively  to  this  question.  For  example,  one  respondent  indi¬ 
cated  that  too  many  CPCIs  were  selected}  and  as  a  result, 
excessive  costs  were  incurred  for  documentation,  configura¬ 
tion  control,  reviews,  ECFs  and  test  plans/procedures. 

A  more  basic  problem  was  highlighted  by  one  Government 
expert  who  stated, 

"The  CPCI  structure  was  meaningless.  It 
did  not  represent  a  meaningful  decompo¬ 
sition  of  the  system.  CPCIs  violated 
the  intent  of  AFR  65-3  (i.e.,  they  did 
not  meet  the  ’end-use  function’  crite¬ 
rion).  As  a  result,  a  CPCI  that  worked 
meant  nothing  in  terms  of  system  per¬ 
formance;  hence,  they  were  useless  in 
obtaining  management  visibility." 

In  addition,  several  respondents  did  not  consider  size  to  be 

a  primary  concern  in  CPCI  selection.  One  contractor  expert 

made  this  point  clear  by  stating,  "Size  is  only  used  by  the 

technical  man  who  doesn’t  concern  himself  with  the  contract." 


of  those  responding  to  this  question  indicated  that  they 
were  familiar  with  the  criteria  used  for  allocating  system 
requirements  to  CPCIs  on  their  present  (past)  program.  In 
addition,  they  listed  many  of  the  criteria.  Those  most  often 
cited  by  the  respondents  were? 

a.  "Functional  modularity  (i.e.,  grouping  like  func¬ 
tions)  is  a  necessity." 

b.  "Software  developed  by  separate  contractors/vendors 
shall  be  separate  CPCIs." 

c.  "Support  software  shall  be  a  separate  CPCI." 

d.  "Each  CPCI  shall  operate  on  a  single  computer." 

e.  "Minimize  the  number  of  CPCIs  and  interfaces." 

f.  "Keep  the  complex  interfaces  within  a  CPCI." 

g.  "On-line  and  off-line  diagnostics  shall  be  separate 
CPCIs." 

h.  "Similar  requirements  and  functions  that  operate 
interactively  shall  be  in  the  same  CPCI." 

i.  "Off-the-shelf  and  developmental  software  must  be 
separate  CPCIs. " 

j.  "Separate  locations  and  functions  require  separate 
CPCIs." 

In  addition,  some  other  considerations  were  mentioned  less 
frequently  by  the  respondents.  They  are  summarized  below i 

a.  "It  should  be  possible  to  effectively  test  a  CPCI 
as  a  whole.  " 

b.  "Selection  should  be  based  upon  the  CPCI  end-item 
support  requirement  (i.e.,  method  of  distribution)." 

c.  "Allocate  on  the  basis  of  system  performance  param¬ 
eters,  such  as  MADT(maximum  allowable  down-time)  and 
Pt (processing  time)." 


125 


d.  "Politics  among  the  contractors,  procuring  agency, 
using  organization ,  and  maintenance  organization 
must  be  considered." 

e.  "Previous  experience  with  a  similar  type  of  system 
is  desirable." 

f .  "User  and  maintainer  advocacy  must  be  considered. " 

...  -  g.  "Documentation  requirements  should  be  minimized." 

h.  "Separate  users/deployment  phase  control  should 
*  signal  separate  CICIs." 

•  How  Were  the  Criteria  Selected?  Although  almost  half 

of  the  respondents  were  unfamiliar  with  how  the  CFCI  selec¬ 
tion  criteria  were  chosen  for  their  program,  a  wide  variety 
of  answers  were  received  for  this  question.  They  ranged 
from  "the  contractor  did  it"  to  "the  Government  imposed 
them,"  with  "previous  experience"  being  the  most  often  cited 
response.  In  addition,  the  following  considerations  were 
identified : 

a.  "Comparisons  with  capabilities  of  the  existing 
systems " 

b.  "Efficiency  and  ease  of  documentation" 

c.  "Contractual  responsibilities" 

d.  "Delivery  schedules" 

e.  "Transfer  and  turnover" 

f.  "Common  data  base  usage" 

g.  "Number  of  design  reviews  and  audits" 

h.  "Requirements  traceability  for  testing" 

i.  "AFLC  responsibility  for  the  operating  system  and 
off-line  diagnostics" 

j.  "Vendor  documentation  copyrights  and  updates" 
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Despite  the  listing  of  some  criteria  in  the  DoD  and  AF 
policies  and  procedures  on  software  acquisition,  only  two  re¬ 
sponses  mentioned  these  guidelines.  On  the  other  hand, 
several  non-technical  responses  were  received.  They  in¬ 
cluded,  "common  sense,"  "uneducated  guess,"  "kludge,"  "oy 
default  due  to  inexperience  and  lack  of  software  acquisition 
understanding,"  "'blue  sky'  and  'back  of  envelope’  calcula¬ 
tions,"  and  "judgement  of  the  people  preparing  (or  managing) 

t 

each  system  segment  specification, " 

Advantages  and  Disadvantages  of  These  Criteria?  Sev¬ 
eral  advantages  and  disadvantages  of  these  CPCI  selection 
criteria  were  listed  by  the  respondents.  They  were  cate¬ 
gorized  by  the  method  of  their  selection.  In  general,  there 
was  agreement  that  those  system  requirements  allocated  by 
the  contractors  resulted  in  a  total  collection  of  system 
software  that  can  be  managed  in  accordance  with  established 
Air  Force  procedures.  Although  the  number  of  respondents  in 
this  category  was  small  and  primarily  consisted  of  contrac¬ 
tors,  several  specific  advantages  were  described.  They  in¬ 
cluded  the  following: 

a.  "The  Government  can  hold  the  contractor  responsible 
for  a  totally  integrated  design  package." 

b.  "A  complete  package  is  available  at  acceptance. “ 

c.  "Cost  to  the  Government  is  minimized." 

d.  "The  Government  does  not  accept  responsibility  pre¬ 
maturely.  " 

e.  "Programmer  assignment  is  easier." 
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f.  "Unit  testing  is  easier." 

g.  "Documenting  the  system  is  easier." 

h.  "The  overall  system  is  easier  to  define." 

On  the  other  hand,  one  respondent  (a  contractor)  stated, 
"Proper  application  (of  the  selected  criteria)  sometimes 
requires  more  knowledge  and  experience  in  system  acquisition 
«  concepts  and  implications  than  is  always  brought  to  bear." 

*  This  problem  was  supported  by  another  respondent  who  asserted 
that,  "The  criteria  are  not  sufficiently  documented  in  to¬ 
day's  MIL-3PECs  and  are  not  understood  by  the  personnel 
doing  the  selection. " 

For  the  "Government-imposed"  criteria,  the  disadvan¬ 
tages  appear  to  greatly  outweigh  the  advantages.  In  other 
words,  only  three  minor  advantages  were  discernable  by  the 
respondents.  They  were i 

a.  "Traceability  of  requirements  to  design  was  made 
easier, " 

b.  "CPC Is  were  more  manageable  during  development  and 
test, "  and 

c.  "line  item  delivery  was  possible." 

Conversely,  several  respondents  described  drastic  results 

•  which  occurred  when  CPCI  structures  were  imposed  upon  con¬ 
tractors.  For  example,  on  one  program,  "The  system  level 

9 

specification  submitted  to  prospective  bidders  specified  the 
computer  program  components (CPCs ) ,  as  well  as  the  CPCIs.  As 
a  result,  several  engineering  change  proposals (ECPs)  were 
generated  and  submitted  by  the  eventual  bid  winner. "  In 
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another  case,  "Completion  of  the  FQT(functional  qualifica¬ 
tion  test)  did  not  allow  observation  of  a  working  portion  of 
the  system.  Also,  the  structure,  when  combined  with  the 
APR  800-14  definition  of  allocated  baseline,  caused  the  SPO 
(systems  program  office)  to  focus  on  design  detail  and  for¬ 
get  system  performance." 

Even  though  experience  was  the  most  often  used  method¬ 
ology  for  choosing  CPCI  selection  criteria,  the  respondents 
did  not  agree  on  its  utility.  For  example,  one  respondent 
stated,  "Experience  may  lead  to  selection  of  CPCI  content 
that  is  forced  into  an  unnatural  mold  because  'it  was  done 
this  way  before'  or  because  the  people  in  charge  have  no 
knowledge  of  better  techniques. "  Another  respondent  indi¬ 
cated  that  CPCI  selection  criteria  chosen  on  the  basis  of 
experience  are  hard  to  clarify,  and  often  overlap.  This  re¬ 
sults  in  "independent  systems,  duplicated  functions,  and 
difficult  integration  problems."  On  the  other  hand,  crite¬ 
ria  chosen  in  this  manner  do  "minimize  specification  main¬ 
tenance  problems  (i.e.,  ECls),  testing/requirements  trace- 
ability  problems,  and  turnover  problems." 

The  two  respondents  who  reported  on  criteria  selected 
from  the  DoD  and  AF  policies  and  procedures  (i.e.,  MIL-STD- 
483  and  an  E3D  Guidebook)  stated  that  the  criteria  were 
"understandable  and  worked  for  most  applications"!  but  "may¬ 
be  all  configuration  contingencies  were  not  considered. " 


Criteria  Different  from  Those  Previously  Used  (Ql6)? 

Almost  kO°A  of  those  Interviewed  failed  to  respond  to  this 

question j  and  those  who  did  were  evenly  split  among  positive 

and  negative  responses.  In  most  cases  that  were  different, 

the  previously  used  criterion  was  to  allocate  requirements 

on  the  basis  of  functional  modularity.  For  example,  CPCIs 

were  defined  for  utility,  control,  application,  simulation, 

and  data  reduction  functions.  However,  one  respondent,  who 

had  18  years  of  experience  in  software  development,  stated, 

"In  two  (previous)  cases,  CFO  advisors 
insisted  that  'size*  and  'visibility* 
were  criteria  to  be  applied  with  priority 
over  all  others.  Generally,  those  par¬ 
ticular  advisors  happened  to  be  software- 
technical  people  with  limited  knowledge/ 
experience  in  (1)  the  Air  Force  and  (2) 
defense  system  program.  They  did  not 
really  understand  the  configuration  man¬ 
agement  processes  of  identification,  con¬ 
trol,  and  status  keeping,  nor  the  ways 
in  which  those  processes  depend  funda¬ 
mentally  on  sound  principles  of  Cl  se¬ 
lection,  Both  of  those  programs  proved 
to  be  recognized  failures.  While  there 
were  other  factors,  in  both  cases,  that 
approach  to  CiCI  selection  was  suffi¬ 
cient  in  itself  to  ensure  that  serious 
troubles  would  be  encountered,  down¬ 
stream.  " 

Advantages  and  Disadvantages  of  Those  Freviouslv  Used 
Criteria?  Although  CPCIs  defined  on  the  basis  of  functional 
modularity  generally  provide  the  manager  with  a  "high  level 
of  visibility  during  the  early  development  stages,"  this 
criterion  has  some  significant  disadvantages.  They  include « 

a.  "Documentation  is  a  nightmare  (e.g. ,  many  ECPs  in 
the  inter-CPC  I  interface  area), 
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b.  traceability  of  functional  requirements  to  design 
is  poor,  and 

c.  suboptimization  of  design  at  the  CPCI  level  is  en¬ 
couraged  (e.g. ,  each  CHJI  may  satisfy  its  allocated 
requirements  but  software  integration  is  still  an 
extended  process)." 

Anv  Other  Criteria  That  Should  Be  Used  (317)?  Over  60/i 
of  those  responding  to  this  question  identified  other  crite¬ 
ria  or  considerations  that  they  believe  should  be  used  in 
decomposing  the  system  requirements.  They  are  summarized  as 
follows « 

a.  "Organizational  interfaces  associated  with  the  de¬ 
velopment  team  must  be  considered.  Ideally,  the 
development  team  should  be  organized  about  the 
functional  requirements  of  the  jobj  but  seldom  is 
this  ideal  achieved.  " 

b.  "Interfaces  between  CTCIs  should  be  as  simple  and 
stable  as  possible.  If  the  interface  is  really  a 
hairy  one;  then,  perhaps,  the  CPC  Is  on  each  side  of 
the  interface  should  be  done  by  the  same  contractor 
or  the  CPC  Is  should  be  made  a  single  CPC  I  thus 
eliminating  the  interface." 

c.  "In  a  distributed  processing  environment,  where 
microprocessors  implement  individual  functions, 
there  is  a  strong  case  for  identifying  CPC  Is  in  ac¬ 
cordance  with  the  specific  processor  (computer)  in 
which  it  will  operate.  This  would  result  in  a 
•cleaner, '  more  manageable  effort,  especially  in 
documentation. " 

d.  "The  future  organizational  structure  of  the  user 
and  maintainer  should  be  considered.  In  other 
words,  separate  CPC  Is  are  required  if  the  future 
maintenance  organization  for  the  mission  software 
is  not  the  same  as  that  for  the  support  software." 

e.  "More  careful  consideration  of  the  contractual  and 
legal  implications  of  a  'CPCI'  are  required.  Re¬ 
spondents  from  both  the  Government  and  industry 
voiced  this  concern  in  their  response  to  this  ques¬ 
tion.  For  example,  one  Government  expert  described 
the  following  scenario.  "We  could  impose  the  full 
vigor  of  APR  800-14  and  be  in  a  position  of  having 
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to  accept  individual  CK)ls  which  satisfy  the  indi¬ 
vidual  specifications  and  tests,  but  whicn  when  in¬ 
tegrated  among  themselves  and  with  the  rest  of  the 
system  could  require  major  rework  to  achieve  system 
performance.  Uur  tendency  has  been  to  hold  con¬ 
tractors  to  system  performance  but  still  require 
CICIs.  This  seems  like  a  contradiction  and  prob¬ 
ably  is.  When  a  project  goes  well,  no  one  cares 
about  the  contradiction!  however,  when  a  project 
goes  to  'disasterville, *  there  seems  to  be  the  legal 
argument  that  the  AF  owns  the  CIs,  thus  it  is  not 
the  fault  of  the  contractor  that  he  cannot  inte¬ 
grate  them  to  achieve  system  performance." 

f.  "Software  requirements  must  evolve  along  system 
functional  lines  to  minimize  the  communications  gap 
between  management,  system  engineering,  and  software 
engineering.  The  iterative  evolution  is  important 
when  the  system  is  different  from  the  teams  pre¬ 
vious  experience." 

g.  "More  emphasis  on  software  maintenance  is  requireu. 
Good  on-line  and  off-line  diagnostics  must  be  spec¬ 
ified  (as  CPCIs),  In  summary,  a  contractor  who  de¬ 
signs  (structures)  a  system  for  easy  software  main¬ 
tenance  will  have  a  much  easier  time  in  the  inte¬ 
gration  phase." 

h.  "More  involvement  of  programmers  in  system  design  is 
desirable.  " 

i.  "A  system  must  be  decomposed  into  ' subsystems ' -not 
in  the  sense  that  a  DBMS  is  a  subsystem,  but  in  the 
sense  that  if  the  program  were  cancelled  right  now, 
would  the  CPCIs  that  have  passed  FQT  give  the  user 
the  ability  to  do  some  portion  of  his  mission. 

CPCIs  structured  around  (functional)  system  tasks 
(observation  collection,  math  subroutines,  etc.) 
would  not. " 

In  summary,  one  FCRC  expert,  who  had  25  years  of  experience 
in  the  development/acquisition  of  computer  software,  stated, 
"This  lessons-learned  set  of  information  is  needed  and  may 
only  come  as  a  composite  of  responses  to  your  survey." 

Criteria  for  CPCIs  Different  From  CIs  (Ql8)?  Slightly 
over  half  of  those  responding  to  this  question  indicated 
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that  the  criteria  used  for  selecting  CTCIs  are  different 
from  those  used  for  allocating  system  requirements  into  CIs. 
One  respondent  summarized  the  difference  by  stating,  "CTCIs 
and  CIs  are  both  a  level  of  delivery  on  a  contract,  but  CIs 
tend  to  be  delineated  by  physical  criteria  more  so  than 
functional  or  logical  considerations."  In  other  words, 
"Software  hasn't  concerned  itself  with  F^(form,  fit,  and 
function),  interface  standardization,  etc.  like  hardware 
has.  As  a  result,  there  are  no  known  MIL-STDs  for  software 
system  design  or  software  modules  as  there  are  for  hardware." 
In  summary,  "The  basic  objectives  and  essential  considera¬ 
tions  are  the  same  for  CIs  and  CTCIs.  However,  there  is 
more  flexibility  in  the  case  of  CTCIs,  due  to  the  absence  of 
requirements  associated  with  production/manufacturing,  spare 
parts  control,  etc." 

Experienced  Any  Problems  With  the  C1CI  Structure  JMl? 
Over  7 2%  of  those  responding  to  this  question  indicated  that 
they  had  experienced  problems  with  the  CTCI  structure  on  the 
program  they  evaluated.  In  other  words,  evidence  exists  to 
support  the  hypothesis  that  problems  are  experienced  with 
the  CTCI  structure  on  more  than  half  of  the  system  develop¬ 
ment/acquisition  programs  that  are  software-intensive  (Ref. 
Appendix  L  for  t  test  results).  Another  interesting  statis¬ 
tical  inference  is  that  problems  with  the  CTCI  structure  be¬ 
come  more  prevalent  as  a  program  gets  larger.  This  was  in¬ 
dicated  by  the  simple  bivariate  correlation  of  -0.55  between 
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this  question  and  program  size  in  terms  of  number  of  lines 
of  code. 

As  reported  by  several  respondents,  the  impact  of  these 
problems  can  be  devastating  to  a  system  development/acquisi¬ 
tion  program.  In  many  cases,  "documentation  becomes  con¬ 
fusing,  requirements  are  'lost,'  schedules  are  slipped, 
users  get  frustrated,  and  CICIs  are  delivered  with  an  enor¬ 
mous  number  of  software  discrepancies."  The  solutions  to 
these  problems  have  been,  in  some  cases,  as  drastic  as  the 
problem  itself.  For  example,  one  "program  was  cancelled"  and 
in  another  one  "the  schedule  slipped  13  months,  costs  bal¬ 
looned,  and  testing  requirements  were  relaxed."  However,  in 
most  cases,  "ECFs  are  generated  to  modify  the  initial  CPCI 
list  and  thousands  of  difficult  software  fixes  are  performed." 

Alternative  CPC  I  Structures  Evaluated  (040)?  Over  half 
of  those  responding  to  this  question  did  not  believe  that  an 
adequate  analysis  of  the  alternative  CPCI  structures  was 
performed  on  their  program.  The  co**<nents  supporting  this 
result  ranged  fro®,  "There  were  no  real  alternatives  consid- 
ered-we  should  have  but  we  didn’t,"  to  "In  the  framework  of 
system  programs  managed  in  accordance  with  established  Air 
Force  policy,  there  are  no  alternatives."  However,  the  ma¬ 
jority  of  those  responding  negatively  identified  preliminary 
system  design  as  the  area  needing  additional  emphasis.  For 
example,  "More  technical,  management,  and  operational  plan¬ 
ning  are  required  before  CPCI  boundaries  are  fixed."  One 
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Government  expert  clearly  described  this  need  by  stating, 

"The  current  thrust  of  analysis  (per  AFR  800-14)  is  hard¬ 
ware  oriented  (e.g.,  through-put,  core,  timing,  etc,),  rather 
than  oriented  towards  a  logical  decomposition  of  the  system 
into  manageable  units,  which  if  managed  properly,  will  re¬ 
sult  in  a  working  end-use  system  function  vice  an  internal 
non-stand  alone  system  task. " 

The  simple  bivariate  correlations  between  the  responses 
to  this  question  and  the  other  program  characteristics  ques¬ 
tions  produced  some  interesting  inferences.  First  of  all, 
the  respondents*  opinion  on  the  adequacy  of  the  analysis  of 
alternative  CPC  I  structures  was  negatively  correlated  with 
program  size,  both  in  terms  of  number  of  CPCIs  (-0.37)  and 
number  of  lines  of  code  (-0.43).  In  other  words,  those  per¬ 
sonnel  evaluating  large  software-intensive  programs  tended 
to  believe  that  an  adequate  analysis  of  the  alternative  CPCI 
structures  was  not  performed  on  their  program.  However,  if 
a  respondent  believed  that  an  adequate  analysis  of  the  al¬ 
ternatives  was  performed  on  his  program,  he  tended  to  agree 
that  the  CPCIs  were  all  defined  at  about  the  right  size  in 
terms  of  managing  their  development.  This  was  indicated  by 
a  positive  correlation  of  0.48  between  these  two  questions. 

In  another  case,  the  respondents'  opinions  on  the  sim¬ 
ilarities  of  Cl  and  CPCI  selection  criteria  were  negatively 
correlated  with  their  opinions  on  the  adequacy  of  the  anal¬ 
ysis  of  alternative  CPCI  structures  (-0.57).  However,  the 
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most  significant  inference  was  that  an  inadequate  analysis 
of  the  alternative  CPC  I  structures  leads  to  problems  with 
the  chosen  CPC  I  structure.  This  was  derived  from  a  negative 
correlation  of  -0.52  between  the  respondents'  opinions  on  the 
adequacy  of  the  analysis  of  alternative  CPCI  structures  and 
whether  they  had  experienced  any  problems  with  the  CPCI 
structure  on  their  programs. 

Suggested  Changes  in  the  Way  CiCIs  Are  Selected.  Many 
changes  in  the  way  CPCIs  are  selected  were  suggested  oy  those 
interviewed.  Most  of  the  suggestions  involved  procedural 
concerns  and/or  the  level  of  emphasis  placed  on  this  process. 
In  addition,  several  respondents  discussed  the  problems  as¬ 
sociated  with  selecting  CPCIs  on  a  software-intensive  system 
without  making  any  specific  recommendations.  One  FCRC  ex¬ 
pert  stated, 

"The  CPCI  structure  is  established  too 
early  in  the  acquisition  cycle.  More 
pre-RFI’  analysis  is  needed.  More  time 
is  needed  for  government  analysis  before 
releasing  contracts.  The  SDR,  PDR,  and 
CDR  schedules  must  be  slowed  down  to 
prevent  the  current  practice  of  starting 
to  code  before  PDR  is  over  (SDR  90  days 
after  contract  award  is  silly).  We  rush 
too  much  early  in  the  game,  don't  test 
enough j  and  pay  dearly  for  it  later.  We 
always  do-we  never  learn l " 

This  opinion  was  supported  by  several  government  and  con¬ 
tractor  experts  who  indicated  the  need  for  more  up-front 
systems  planning  and  an  evolutionary  approach  to  the  develop¬ 


ment  of  large  software  projects. 

The  other  important  but  less  frequently  mentioned 


changes  were  as  follows i 


a.  "Isolation  of  vender  software  into  separate  CPCIs 
should  be  mandatory.  *’ 

b.  "Agencies  responsible  for  the  software  maintenance 
after  turnover  should  be  identified  and  considered." 

c.  "More  emphasis  on  the  system  product  after  integra¬ 
tion  and  less  emphasis  on  individual  CPCI  develop¬ 
ment  visibility  is  needed." 

d.  "The  problems  of  common/adapted  software  documenta¬ 
tion  needs  more  emphasis. " 

e.  "The  using  and  supporting  activity  must  be  involved 
beginning  at  the  time  the  system  is  determined  to 
become  digital/computerized,  and  they  should  have 
responsibility,  with  the  contractor,  for  CPCI  se¬ 
lection.  " 

f.  "There  is  an  urgent  and  widespread  need  for  better 
training  of  both  SPO  and  contractor  personnel  in 
the  meaning,  purposes,  functions,  and  management 
implications  of  the  CPCI  concept." 

g.  "CPC Is  should  be  selected  on  a  functional  string 
basis  so  that  CPCI  interrelationships  are  mini¬ 
mized.  " 

h.  "The  discrepancies  in  AFR  800-14  and  MIL-STD-483 
regarding  definition  of  the  allocated  baseline  need 
to  be  resolved.  The  allocated  baseline  is  supposed 
to  be  a  'design  to’  baseline,  yet  the  content  re¬ 
quired  by  these  documents  is  detailed  design  infor¬ 
mation.  AFR  800-14  says  'math  models'  should  go  in 
Section  6  of  the  Part  I  specification  (which  is  not 
contractually  binding),  then  says  'algorithms' 
should  be  in  Section  3.  What  is  the  difference j 
they  should  all  be  in  Section  6,  if  in  the  Part  I 
at  all." 

Suggestions  for  Improving  the  Software  Requirements 
Decomposition  Process  (Q4l).  This  question  provoked  a  wide 
variety  of  responses,  which  were  categorized  into  fiv  ■ 
areas >  visibility,  structured  approach,  lack  of  understand¬ 
ing,  expertise  required,  and  flexibility.  As  for  visibility, 
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one  respondent  stated, 

"Cl Cl  selection  must  be  made  bearing  in 
mind  that  the  CIO  is  seeking  visibility 
into  contractural  processes,  and  the 
CPC  I  structure  is  one  tool  to  provide 
visibility.  It  must  be  recognized  that 
meaningful  visibility  is  not  necessarily 
gained  by  taking  ’vertical  snapshots' 
of  the  system.  u,ach  facet  of  require¬ 
ments  decomposition  must  be  accompanied 
by  the  questions-will  this  particular 
entity  give  me  visibility,  will  sub¬ 
dividing  more  give  me  more  visibility, 
and  if  each  unit  (CPCI)  passes  its  FQT, 
will  I  have  a  complete  (in  the  'hori¬ 
zontal'  sense)  portion  of  the  system  up 
and  running?" 

Several  respondents  discussed  the  need  for  a  more  struc 
tured  approach  to  the  decomposition  process.  Both  require¬ 
ments  definition  methodologies  and  design  techniques  were  in 
eluded.  For  example,  system  simulation,  top-down  modular  de 
sign,  step-wise  refinement,  structured  Analysis  and  Design 
Technique (3ADT) ,  Computer  Aided  Design  and  System  Analysis 
Tool(CADSAT) ,  and  computerized  text  editing  were  all  sug¬ 
gested  as  tools  that  would  help  to  better  clarify  the  func¬ 
tions  and  their  interrelationships. 

Although  the  concept  of  partitioning  the  requirements 
of  a  system  into  subsystems,  and  a  subsystem  into  components 
has  been  in  use  for  many  years,  several  respondents  to  this 
question  believe  that  a  general  lack  of  understanding  exists 
both  in  the  Government  and  in  the  computer  industry,  A  well 
known  contractor  expert,  who  had  25  years  of  experience  in 
the  development/acquisition  of  software,  stated, 
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HI  believe  we  have  taken  a  major  step 
backwards,  since  1969  (when  the  decision 
was  made  to  establish  a  standard  approach 
for  all  three  services).  The  goal  was 
an  admirable  one  but  the  results  have 
not  been  that  great.  Having  to  re-write 
the  MIL-SPSCs  and  get  tri-service  agree¬ 
ment  on  them  has  resulted  in  all  the  de¬ 
tail  being  removed.  Personnel  entering 
the  field  since  1969  have  great  diffi¬ 
culty  in  understanding  the  process  if 
they  have  to  rely  on  today's  MIL- 3 PECs 
and  regulations.  The  MIL-SPECs  assume 
that  the  reading  audience  has  an  in 
depth  level  of  experience  and  knowledge 
which  is  non-existant  in  both  government 
and  industry." 

Several  approaches  to  minimizing  the  impact  of  this 
situation  were  suggested  by  the  respondents.  They  included 
the  following  suggestions: 

a.  "Develop  a  good  example  of  the  requirements  decom¬ 
position  process  and  its  documentation  and  make  it 
available  to  all  concerned  parties." 

b.  "Fund  the  development  of  an  acquisition  guidebook, 
and  incorporate  the  guidebook  information  in  one  of 
the  AFSC  800-series  pamphlets. " 

c.  "Publish  the  results  of  this  study  of  the  process 
and  make  it  available  to  the  software  industry,  as 
well  as  those  in  the  Government  (procurers,  devel¬ 
opers,  and  users). 

d.  "Select  a  contractor  who  has  people  with  a  proven 
track  record.  Quit  bringing  in  line  pilots  and 
navigators  to  run  software  programs.  Let  the  peo¬ 
ple  who  have  done  it  and  learned  from  their  mis¬ 
takes,  when  they  were  young,  try  it.  Use  top-down 
approach.  Get  something  working,  then  add  capabil¬ 
ity.  Hire  companies  that  have  a  profit  motive  to 
perform  analyses  and  studies,  rather  than  an  FCRC 
organization. " 

e.  "Use  personnel  who  have  expertise  in  four  areas: 

(1)  The  prime  functions  allocated  to  the  system 
whose  CPC  I  requirements  are  being  decomposed. 
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(2)  Associated  functions  of  the  hardware  CIs  of 
the  system. 

(3)  Immediately  relevant  functions  of  interfaced 
systems. 

(4)  The  operational  capabilities  which  will  be  pro¬ 
vided  by  the  system  and  how  they  will  be  em¬ 
ployed.  " 

f.  "Provide  more  and  better  training  of  software  en¬ 
gineers  involved  in  system  programs.  Ensure  more 

4  personnel  are  trained  in  the  support  management 

disciplines  (notably,  configuration  management) 

*  with  respect  to  principles  and  procedures  that 

apply  specifically  to  software. " 

Finally,  flexibility  seems  to  be  a  necessity,  especially 
in  developing  any  real-time  system  such  as  a  system.  In 
other  words,  "Requirements  are  going  to  change  and  enough 
flexibility  should  be  planned  into  the  project  to  allow  for 
justified  changes."  This  includes  schedule  and  budget,  as 
well  as  the  system  architecture. 

Evaluation  of  Horizontal  Allocation 

Section  III  of  the  interview  questionnaire  asked  the 
expert's  opinion  on  the  use  of  horizontal  allocation  as  a 
methodology  for  allocating  software  requirements  to  CPCIs. 

To  ensure  a  common  understanding  among  those  interviewed,  a 

*  definition  of  the  concept  and  a  simple  example  were  included 
with  each  questionnaire.  Then,  five  questions  addressed  the 
feasibility,  potential  impact  on  12  parameters,  advantages/ 
disadvantages,  and  implementation  of  the  concept.  The  sta¬ 
tistical  results  (tabulated  in  Appendices  I  and  J)  and  qual¬ 
itative  responses  for  each  question  are  summarized  in  the 
following  paragraphs. 
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Feasibility  (Q42)?  Over  82%  of  those  responding  to  this 
question  indicated  that  horizontal  allocation  is  a  feasible 
methodology  for  allocating  software  requirements  to  CPCIs  on 
a  large  software  development  project.  This  is  sufficient 
evidence  to  statistically  support  the  hypothesis  that  more 
than  half  of  the  software  development/acquisition  experts  in 
Government  and  industry  believe  that  horizontal  allocation 
is  feasible  as  defined  (Ref.  Appendix  L  for  t  test  results). 

The  analysis  of  variable  interdependency  between  this 
question  and  those  nominal-scale  variables  identified  in 
Appendix  F  (using  the  Chi-square  test)  resulted  in  no  sta¬ 
tistically  significant  correlations  at  or  below  the  0.05 
level  (Ref.  Table  XVII  of  Appendix  J). 

On  the  other  hand,  an  analysis  of  the  bivariate  corre¬ 
lations  between  this  question  and  those  interval-scaled  pro¬ 
gram  characteristic  questions  produced  some  interesting  in¬ 
terdependencies  (Ref.  Table  XV  of  Appendix  J).  For  example, 
the  size  of  the  program  (in  terms  of  number  of  CPC  Is  and 
lines  of  code)  was  positively  correlated  with  the  respon¬ 
dent's  opinion  on  the  feasibility  of  horizontal  allocation. 
Likewise,  those  respondents  who  experienced  problems  with 
the  CPCI  structure  on  their  program  were  more  likely  to 
agree  that  the  methodology  is  feasible. 

Similarly,  the  Pearson  Product-Moment  Correlations  be¬ 
tween  the  answers  to  this  feasibility  question  and  the  per¬ 
ceived  effectiveness  questions  were  all  positive  and 
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statistically  significant,  except  for  one  (Ref.  Table  XVI 
of  Appendix  J).  That  is,  the  respondents’  opinions  on  the 
feasibility  of  horizontal  allocation  and  how  effective  it 
would  be  in  reducing  the  software  integration  task  were  not 
significantly  correlated. 

Perceived  Effectiveness  (Q41-Q54).  This  question  re¬ 
quested  the  respondent  to  evaluate  the  horizontal  allocation 
methodology  on  the  basis  of  its  impact  on  12  parameters. 

Each  response  was  scored  on  a  simple  1  to  4  scale,  where  1 
equals  "not  effective,"  2  equals  "somewhat  effective,"  3 
equals  "moderately  effective,"  and  4  equals  "very  effective," 
The  statistical  results  for  each  of  these  parameters  are 
tabulated  in  Appendices  I,  J,  and  L.  In  addition,  the  re¬ 
sults  are  briefly  discussed  in  the  following  paragraphs. 

An  analysis  of  the  responses  to  these  questions  pro¬ 
vides  two  interesting  results.  First,  a  large  majority  of 
those  responding  to  these  questions  indicated  that  the  hori¬ 
zontal  allocation  of  software  requirements  to  CICIs  would 
favorably  impact  all  but  one  of  the  12  parameters  (Ref. 

Table  VIII  of  Appendix  I).  In  fact,  there  was  sufficient 
statistical  evidence  to  support  the  hypothesis  that  more 
than  half  of  the  Government  and  contractor  experts  believe 
that  horizontal  allocation  would  be  at  least  somewhat  ef¬ 
fective  in  favorably  impacting  all  the  parameters,  except 
for  complexity  of  the  decomposition  process  (Ref.  Table  XIX 
of  Appendix  L  for  t  test  results).  In  addition,  over  60% 
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of  the  respondents  indicated  that  this  approach  to  CPC  I  se¬ 
lection  would  be  moderately  to  very  effective  in t 

a.  providing  the  program  manager  and  user  with  more 
objective  visibility  into  the  system  development, 

b.  improving  the  software  quality  (e.g.,  fewer  latent 
errors), 

c.  reducing  the  software  integration  task,  and 

d.  highlighting  problems  earlier. 

These  results  were  supported  with  written  comments  from 
some  of  the  respondents.  For  example,  one  government  expert 
stated,  "The  user  loses  phony  visibility  and  gains  visibility 
into  a  working  system  through  ’seeing,  touching,  and  feeling* 
the  working  increments."  Another  respondent  wrote,  "This 
approach  to  CPC  I  selection  is  basically  intended  to  provide 
the  user  with  some  initial  operational  capability  instead  of 
having  to  wait  on  some  longer  schedule  for  the  total  opera¬ 
tional  capability."  Software  maintenance  will  also  be  eas¬ 
ier  and  less  costly  according  to  one  government  expert  who 
said,  "The  system  will  have  been  subjected  to  many  more  hours 
of  realistic  testing  prior  to  turnover,  thus  less  errors 
will  remain.  Similarly,  if  the  contractor  is  required  to 
write  a  product  specification  on  each  increment,  and  the 
specification  is  baselined  immediately  after  the  incremental 
FQT,  the  system  will  be  documented  very  well,  which  is  more 
than  can  be  said  for  many  of  today's  operational  systems." 

In  addition,  several  respondents  provided  positive  com¬ 
ments  on  the  effectiveness  of  horizontal  allocation  in 
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highlighting  problems  earlier.  One  government  expert's  re¬ 
sponse  was,  "Definitely.  The  real  problems  of  a  software 
development  (i.e.,  the  software  doesn't  work  as  a  system) 
are  not  discovered  in  today’s  approach  until  after  all  CPCIs 
are  coded  and  FQT’d.  In  this  approach,  system  problems  will 
be  identified  very  early,  and  can  be  diagnosed  on  the  spot." 

On  the  other  hand,  some  comments  indicated  that  hori¬ 
zontal  allocation  could  be  detrimental  to  a  software  devel¬ 
opment  project.  One  respondent  indicated  that,  "This  method¬ 
ology  would  allow  development  to  be  accomplished  in  a  struc¬ 
tured  step-by-step  manner,  rather  than  in  one  giant  leap  as 

v 

we  do  it  today.  Therefore,  the  total  schedule  would  con¬ 
ceptually  be  longer  but  most  likely  shorter  since  this  de¬ 
velopment  approach  would  be  less  risky."  More  specific  com¬ 
ments  were  received  on  the  impacts  this  approach  would  have 
on  software  integration.  For  example,  one  respondent  stated, 
"This  approach  actually  increases  the  amount  of  integration 
work  by  forcing  system  integration  from  the  second  FQT  on." 

Finally,  the  bivariate  correlations  among  the  12  per¬ 
ceived  effectiveness  variables,  and  between  the  perceived 
effectiveness  variables  and  the  demographics/program  char¬ 
acteristics  produced  several  significant  interdependencies. 
For  example,  the  respondents’  opinions  on  the  effectiveness 
of  horizontal  allocation  in  making  debugging  and  testing 
more  efficient,  making  software  maintenance  easier  and  less 
costly,  increasing  the  morale  of  those  working  on  the 
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program,  reducing  the  complexity  of  the  decomposition  pro¬ 
cess,  and  improving  training  effectiveness  were  all  posi¬ 
tively  correlated  with  the  number  of  CFCIs  on  those  programs 
evaluated. 

In  another  case,  the  perceived  effectiveness  of  hori¬ 
zontal  allocation  in  reducing  the  software  integration  task 
was  significantly  correlated  with  the  answers  to  the  ques¬ 
tion,  Have  you  experienced  any  problems  with  the  CFCI  struc¬ 
ture  on  your  present  (past)  programs?  In  other  words,  re¬ 
spondents  who  have  experienced  problems  with  past  CFCI 
structures  believed  that  the  software  integration  task  would 
be  reduced  on  programs  where  CFCIs  are  selected  on  tne  basis 
of  horizontal  allocation.  All  of  these  statistically  sig¬ 
nificant  correlations  are  presented  in  Tables  XII,  XIII,  and 
XIV  of  Appendix  J. 

Adversely  Impact  Contractor  (Q^1?)?  More  than  half  of 
those  responding  to  this  question  indicated  that  this  tech¬ 
nique  would  not  adversely  impact  the  contractor's/developer’s 
management  procedures.  In  addition,  practically  all  of  the 
respondents  provided  a  brief  explanation  of  the  rationale 
for  their  answer.  Several  significant  comments  (both  pro 
and  con)  were  included  in  the  explanations.  Those  indicating 
that  horizontal  allocation  would  adversely  impact  the  con-  . 
tractor/developer  included i 

a.  "A  ’build'  is  a  partial  system.  Therefore,  a  con¬ 
tract  structure  is  needed  to  support  iterative  de¬ 
velopment  and/or  partial  deliveries,  such  as  a 
DD-250/payment  for  each  'build.'" 
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b.  "Division  of  labor  would  be  difficult." 

c.  "The  contractors'  management  procedures  are  ul¬ 
timately  based  on  cost  traceability  with  break¬ 
points  ( IDR/CDR/FQf/System  test)  imposed  oy  the 
Government.  The  pieces  implied  by  this  approach 
are  too  small  for  the  cost  expended  in  tracking  the 
development.  " 

d.  "The  contractor  would  find  this  restriction  on  his 
organizational  freedom  a  handicap  in  properly 
staffing  the  job." 

e.  "The  developer  must  push  himself  to  proceed  to  the 
next  block,  for  it  would  be  easy  to  continue  test¬ 
ing  forever. " 

f.  "There  would  be  a  tendency  to  defer  the  really 
tough  problems  to  future  versions." 

On  the  other  hand,  more  than  half  of  the  comments  in¬ 
dicated  that  this  technique  would  benefit,  not  hinder,  the 
contractor's/developer's  procedures.  Some  of  the  more  spe¬ 
cific  ones  are  listed  below. 

a.  "This  technique  should  improve  the  management  pro¬ 
cedures  by  providing  more  rapid  feedback  on  prog¬ 
ress  as  each  build  is  completed. " 

b.  "Assessment  of  development  status  would  be  alot  more 
obvious. " 

c.  "The  impact  will  be  towards  less  procedure  oriented 
acquisition  and  more  towards  meaningful  acquisition 
milestones  and  realistic  system  integration  and 
test  efforts. " 

d.  "Incremental  development,  which  results  from  this 
approach  to  CiGI  selection,  has  been  used  on  one 
USAF  Strategic  Air  Command (SAC)  program  and  it 
worked.  It  was  implemented  through  a  basic  con¬ 
tract  and  several  options  which  were  to  be  exer¬ 
cised  periodically.  This  approach  gave  the  Govern¬ 
ment  more  control  and  made  the  contractor  more 
responsive. " 

e.  "This  technique  would  force  more  advance  planning 
and  preliminary  design  since  the  CPCI  structure  be¬ 
comes  more  important  contractually.  For  example. 
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the  request  for  proposal (RF1)  would  probably  re¬ 
quire  tr.at  CICIs  be  selected  in  accordance  with 
some  guideline,  standard  or  tecnnical  report}  and 
then  the  Cl  Cl  structure  would  be  a  weighted  evalu¬ 
ation  factor  (both  technical  and  management)  at 
the  Source  Selection  Evaluation  Board (SSEB) . " 

Other  Advantages/Disadvantages  (056)?  Over  80?i  of  those 
responding  to  this  question  identified  some  other  advantages 
and/or  disadvantages  of  selecting  CFCIs  on  the  basis  of  hori¬ 
zontal  allocation.  The  major  advantages  were  stated  as 
follows « 

a.  "This  approach  should  smooth  out  the  use  of  con¬ 
tractor  personnel  and  provide  much  better  continu¬ 
ity  of  the  design/implementation/checkout  process. 
This  leads  to  less  risk  for  the  contractor  and  the 
Air  Force .  " 

b.  "The  clear  advantage  is  flexibility  in  the  develop¬ 
ment,  debug  and  implementation  phases.  Thus,  it 
will  simplify  'crash'  diagnosis,  system  restart, 
and  recovery."  In  addition,  "Operational  use  of 
the  first  build  can  feed  required  operational  modi¬ 
fications  to  subsequent  builds." 

c.  "This  approach,  which  naturally  leads  to  incremental 
development,  provides  more  meaningful  milestones, 
real  visibility  into  contractual  status,  earlier 
user  involvement,  less  meaningless  paperwork,  and 
better  informed  management.  Therefore,  the  visi¬ 
bility  gained  will  not  be  at  the  price  of  causing 
the  acquisition  to  become  bogged  down  in  bureau¬ 
cratic  battles  over  typos  in  Fart  I  specifications. 
In  addition,  this  tecnnique  allows  airtight  con¬ 
figuration  control  of  the  product  as  it  evolves, 
rather  than  paper  intensive,  bogus  control  of  the 
untested  design  via  extant  configuration  management 
procedures.  " 

d.  "It  forces  the  designers  (often  invisible  or  un¬ 
known  to  the  implementer)  to  think  functionally  and 
from  the  start  to  finish." 

e.  "Usually  programs  are  underestimated  in  terms  of 
cost  and  time.  This  method  should  provide  a  better 
idea  of  the  ultimate  cost  of  the  system  and  how 
long  the  development  will  take.  As  a  result,  the 
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Government  may  be  more  likely  to  cancel  a  program 
early,  rather  tnan  trying  to  bail  it  out." 

f.  "This  technique  would  help  functional  test  planning 
and  test  documentation. ”  In  addition,  "The  two 
dimensions  of  test  (capability-does  it  do  the  job?, 
and  performance-now  well  does  it  do  the  job?)  can 
both  be  accomplished. " 

g.  "Although  there  would  seem  that  no  progress  is 
being  made  initially,  it  will  alleviate  the  tradi¬ 
tional  sell-off  crisis." 

The  other  disadvantages  described  by  the  respondents 
were  primarily  concerned  with  obtaining  acceptance  among  the 
Government  and  industry,  and  how  to  contractually  implement 
horizontal  allocation.  For  example,  one  respondent  indicated 
that,  "There  will  be  a  huge  psychological  shift  required  to 
implement  this  approach.  Most  people  do  not  understand. " 

This  point  was  illustrated  by  another  respondent  who  stated, 
"The  disadvantage  appears  to  be  the  design  of  yet  another 
regulation  that  will  be  out  of  date  when  approved.  Circu¬ 
lation  of  data  derived  from  bad  experiences  is  great  and 
should  be  encouraged.  However,  a  regulation  in  this  area, 
particularly  with  regard  to  a  philosophical  rather  than 
purely  technical  viewpoint,  does  not  appeal  to  me." 

In  the  area  of  contractual  implementation,  the  concerns 
expressed  included  the  following! 

a.  "It  may  be  difficult,  if  not  impossible  to  write  a 
fixed  price  contract  for  the  whole  development." 

b.  "Once  you  accept  the  first  model,  you  would  have  to 
GFfi (government-furnished  equipment)  it  back  to  the 
contractor  to  build  the  next  model. " 

c.  "This  approach  may  be  overkill  for  a  job  that  is 
well  understood  and  similar  to  one  previously  per¬ 
formed  by  the  contractor's  team." 
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d.  "The  sole  source  problems  currently  experienced  by 
many  programs  may  become  worse." 

Should  Horizontal  Allocation  Be  Implemented  (Q67)7  Over 
75$  of  those  responding  to  this  question  indicated  that  tnis 
technique  should  be  implemented  on  future  software  develop¬ 
ment/acquisition  programs.  In  other  words,  a  large  majority 
of  the  respondents  believe  that  CPCIs  should  be  defined  in 
terms  of  system  versions  or  models,  each  of  which  contains 
end-use  system  functional  capabilities.  In  fact,  there  was 
sufficient  statistical  evidence  to  support  the  nypotnesis 
that  more  than  half  of  the  Government  and  contractor  experts 
believe  that  this  approach  should  be  implemented  (Ref.  Table 
XIX  of  Appendix  L  for  t  test  results). 

The  analysis  of  variable  interdependencies  produced 
some  interesting  results.  First,  no  statistically  signifi¬ 
cant  interdependencies  (at  or  below  the  .05  level)  were 
found  between  this  question  and  the  nominal  scale  variables 
(Ref.  Table  XVII  of  Appendix  J  for  Chi-square  test  results). 
On  the  other  hand,  the  respondent's  recommendation  on  the  im¬ 
plementation  of  this  technique  was  significantly  correlated 
with  the  number  of  CPCIs  (0.46)  and  the  number  of  lines  of 
code  (0.48)  on  the  program  evaluated  by  the  respondents.  In 
other  words,  respondents  who  evaluated  large  programs  be¬ 
lieved  that  this  approach  should  be  implemented  on  future 
software  development/acquisition  programs. 


Another  interesting  statistical  result  was  that  the  re¬ 
spondent's  opinion  was  significantly  correlated  with  all  the 


perceived  effectiveness  variables  at  or  above  the  0.35  level 
(Ref.  Table  XVI  of  Appendix  J).  The  strongest  relationships 
were  between  the  implementation  question  and  the  respondent's 
opinion  on  the  effectiveness  of  horizontal  allocation  in  re¬ 
ducing  system  development  costs  (O.69),  reducing  system  de¬ 
velopment  time  (0.58),  making  software  maintenance  easier 
and  less  costly  (0.60),  and  increasing  the  morale  of  those 
working  on  the  development  program  (0.58). 

The  respondents  also  provided  several  comments  explain¬ 
ing  their  recommendation  on  the  utility  of  horizontal  allo¬ 
cation.  Some  of  the  comments  were  philosophical  and/or  gen¬ 
eral  in  nature,  while  others  directly  addressed  ways  of  im¬ 
plementing  this  approach.  For  example,  one  respondent 
stated , 

"Management  of  the  development  phase  in 
units,  called  builds  is  great,  and  even 
delivery  in  builds  is  great,  but  after 
delivery,  you  need  to  control  each  com¬ 
ponent  (program)  separately  (e.g.,  op¬ 
erating  system,  application  program, 
data  base,  and  simulator).  In  other 
words,  by  defining  CPC  Is  traditionally 
(Ref.  Figure  ?),  nothing  is  preventing 
the  development/delivery/installation 
of  the  system  in  builds  (Ref.  Figure  8)." 

However,  drastic  problems  often  result  from  this  approach. 

In  other  words,  contracting  for/managing  a  software  develop¬ 
ment  project  on  a  basis  different  from  that  which  the  soft¬ 
ware  is  coded  and  delivered  can  be  devastating  to  the  total 
cost  and  schedule  of  the  system.  In  fact,  on  one  recent  C ^ 
system  acquisition,  the  identity  of  CPCIs  defined  in  the 
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traditional  method  was  lost  when  the  contractor  went  to  the 
"builds"  approach  in  order  to  get  a  meaningful  portion  of  the 
system  working.  This  created  difficult  management  and  tech¬ 
nical  problems  for  the  contractor,  procurer,  user,  and  main- 
tainer.  This  same  result  was  reported  by  several  respondents. 

On  the  other  hand,  one  contractor  expert  indicated  that 
his  company  has  successfully  used  a  top-down  structured  de¬ 
velopment  approach  where  "builds"  were  defined  vertically 
across  all  functions.  Furthermore,  he  stated,  "This  is  the 
best  approach  yet  invented  to  guarantee  a  minimum  of  inte¬ 
gration  problems  at  each  level.  However,  what  is  needed  is 
more  emphasis  on  the  management  of  this  approach."  That  is, 
"Horizontal  allocation  snould  be  encouraged  by  the  standards 
and  regulations  as  a  method  of  defining,  developing,  and  de¬ 
livering  CPC  Is  (Ref.  Figure  9).  but  must  be  left  to  the  con¬ 
tractor  to  propose  if  he  thinks  it  is  appropriate  to  his 
project. " 

Another  expert  stated,  "The  value  of  this  technique  is 
that  it  provides  for  the  necessary  learning  required  by  each 
new  development  team  (i.e.,  one  not  having  done  a  very  simi¬ 
lar  project)."  Thus,  "Horizontal  allocation  should  be  se¬ 
lectively  implemented  on  a  medium  to  small  sized  program  to 
determine  if  fine-tuning  is  required, " 


Conclusions,  and  Recommendations 


Vi.  Gumr.ary . 

This  thesis  documents  a  study  of  the  software  require¬ 
ments  allocation  process  in  the  acquisition  and  management 
of  a  major  defense  system.  Although  the  literature  is  re¬ 
plete  with  studies,  technical  papers,  and  conference  pro¬ 
ceedings  that  discuss  the  proolems  associated  with  software 
engineering  and  software  acquisition  management,  none  of 
them  have  directly  addressed  the  CiCI  selection  process. 
Therefore,  this  study  was  welcomed  by  practically  all  of  the 
Government  and  industry  personnel  contacted  during  the  re¬ 
search  period.  For  example,  one  high-level  Government  ex¬ 
pert  made  the  following  observation,  "If  architects  were  to 
design  buildings  the  way  software  engineers  design  software, 
one  woodpecker  could  destroy  civilization."  In  smother  case, 
an  industry  expert  stated,  "The  present  practice  of  selecting 
CFCIs  is  strictly  a  stab  in  the  dark,  and  it's  about  time 
someone  did  some  research  on  what  has  been  black  magic  for 
too  many  years. " 

The  overall  objective  of  this  study  was  to  determine 
how  the  system  requirements  to  be  implemented  via  software 
are  allocated  to  CPC  Is,  and  to  investigate  the  feasibility 
and  potential  impacts  of  an  alternate  methodology.  An  ex¬ 
tensive  literature  search  and  a  survey  of  Government  and  in¬ 
dustry  "software  experts"  was  used  to  accomplish  this  ob¬ 
jective.  First,  the  DoD  and  AF  policies  and  procedures  on 
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dCI  selection  were  determined  through  the  literature  search. 
Then,  a  semi-structured  interview  questionnaire  was  designed 
to  capture  both  objective  and  subjective  data  on  the  CFCI 
selection  process  currently  used  by  SlOs  and  contractors. 

In  addition,  the  questionnaire  was  used  to  collect  the  ex¬ 
perts'  opinions  on  the  utility  of  horizontal  allocation. 

The  results  were  most  encouraging  and  should  lead  to  a  bet¬ 
ter  understanding  of  this  critical  design  issue.  In  addi¬ 
tion  to  a  summary  of  these  results,  some  conclusions  and  r 
ommendations  are  presented  in  the  following  paragraphs. 

Summary  of  Results 

DoD  and  AF  Policies  and  Procedures  for  CPCI  Selection. 
The  acquisition  and  management  of  computer  resources  embedded 
in  major  defense  systems  is  controlled  by  a  complex  hier¬ 
archy  of  policies  and  directives.  Unfortunately,  only  a  few 
of  the  documents  address  the  process  of  selecting  CFCIs  di¬ 
rectly.  On  the  other  hand,  several  do  present  guidelines 
and  considerations  for  applying  configuration  management 
concepts  to  the  acquisition  and  management  of  software. 

In  summary,  the  DoD  and  AF  policies  on  CFCI  selection 
state,  "Computer  resources  will  be  managed  as  elements  or 
subsystems  of  major  importance  during  the  system  acquisition 
life  cycle."  Furthermore,  "Computer  hardware  and  software 
must  be  specified  and  treated  as  configuration  items." 


These  policies  have  led  to  the  development  of  several  in¬ 
structions,  regulations,  pamphlets,  and  standards.  Although 
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these  documents  in  general  establish  uniform  policies  and 
procedures  for  the  implementation  of  configuration  manage¬ 
ment  within  the  DoD,  the  actual  application  to  software  is 
usually  a  management  decision.  Thus,  several  considerations 
for  use  in  allocating  system  requirements  to  CPCIs  are  pre¬ 
sented  in  these  regulatory  publications.  They  are  as 
follows : 


a.  The  initial  Cl  list  should  contain  more  items  than 
is  anticipated  on  the  final  list. 

b.  £ach  Cl  should  be  produced  and  tested  by  a  single 
contractor  as  an  entity. 

c.  Processes  that  interact  strongly  should  be  assigned 
to  the  same  CPC  I. 

d.  Processes  that  will  execute  in  different  computers 
should  be  allocated  to  different  CPCIs. 

e.  Processes  whose  development  can  feasibly  be  finished 
at  significantly  different  times  should  be  assigned 
to  different  CPCIs. 

f.  Include  in  each  CP'CI  no  more  than  an  individual 
Government  monitor  can  efficiently  track. 

g.  The  life  cycle  cost  and  management  impacts  associ¬ 
ated  with  selecting  CPCIs  should  be  considered 
since  choosing  too  few  or  too  many  can  adversely 
affect  the  program. 

h.  System  trade-offs  and  the  natural  decomposition  of 
the  software  should  be  considered. 

These  policies  and  considerations  have  been  interpreted 
in  a  few  studies,  technical  reports,  conference  proceedings, 
lessons  learned,  and  guidebooks.  Thus,  several  additional 
criteria  and/or  factors  to  be  considered  in  selecting  CPCIs 
were  described  in  Chapter  III  of  this  thesis.  They  are  sum¬ 
marized  as  follows i 
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a.  Each  operational  program  should  be  a  CK)I. 

b.  Premature  "freezing"  of  system  structure  into  man¬ 
agement  structure  results  in  efforts  being  dupli¬ 
cated,  system  integration  becoming  more  difficult, 
and  overall  visibility  being  reduced  as  the  system 
design  evolves. 

c.  Software  tools  should  be  separate  CPCIsj  however, 
resident  debug  programs  are  part  of  the  operational 
CPC  I. 

•  d.  Decompose  the  system  so  that  management  control  and 

flexibility  can  be  maintained. 

• 

e.  lartition  the  system  so  that  implementation,  de¬ 
bugging,  and  testing  of  modules  or  groups  of  modules 
(ClCIs)  can  proceed  in  parallel. 

f.  CPC  Is  should  be  identified  on  the  basis  of  whether 
they  satisfy  an  end-use  function,  are  key  programs, 
have  expected  interface  problems,  and  will  provide 
added  visibility  to  a  particular  computer  program. 

g.  The  CPCI  selection  decision  should  be  made  early  in 
the  program. 

h.  Include  the  AFLC  system  manager  in  the  selection 
process . 

i.  The  number  of  CPCIs  should  be  minimized. 

j.  The  end  result  of  choosing  a  CPCI  as  a  unit  of  man¬ 
agement  should  be  the  ability  to  directly  observe 
that  a  meaningful  portion  of  the  system  works.  Thus, 
a  structure  in  which  the  functional  flow  across  the 
system  is  divided  into  several  CPCIs  is  artificial. 

,  These  interpretations  have  resulted  in  some  discrepan- 

*  cies  between  the  regulations  and  MIL-STDs.  For  example,  the 

A3 PR  and  AF3CP  800-3  consider  computer  software  to  be  an  item 

ft 

of  data  in  the  same  manner  as  reports,  forms,  manuals,  and 
specifications  are.  However,  the  Air  Force  policy,  as  stated 
in  AFSCRP  800-1,  is  that  computer  software  must  be  subjected 
to  the  same  contractual  and  cost  controls  as  hardware. 


157 


Similarly,  AFR  800-14  and  YIL-3TD-483  do  not  agree  on  the 
level  of  detail  required  in  the  Fart  I  specification  for 
software . 

Selection  Criteria  Currently  in  Use.  Over  70%  of  those 
"software  experts"  participating  in  the  survey  indicated 
that  they  were  familiar  with  tne  criteria  used  for  allocat¬ 
ing  system  requirements  to  CPC  Is  on  their  program.  Those 
most  often  cited  by  the  respondents  were  as  follows: 

a.  Software  developed  by  separate  contractors/vendors 
must  be  separate  CICIs. 

b.  Functional  modularity  (i.e.,  grouping  like  func¬ 
tions)  is  a  necessity. 

c.  On-line  and  off-line  diagnostics  and  other  support 
software  must  be  separate  CICIs. 

d.  Each  CPCI  must  operate  on  a  single  computer. 

e.  The  number  of  CPC  Is  and  interfaces  must  be  minimized. 

f.  Keep  the  complex  interfaces  within  a  CPCI. 

g.  Off-the-shelf  and  developmental  software  must  be 
separate  CPCIs. 

h.  Separate  locations  and  functions  require  separate 
CICIs. 

In  addition,  some  other  considerations  were  identified 
by  the  respondents.  They  included  the  following: 

a.  It  should  be  possible  to  effectively  test  a  CPCI 
as  a  whole. 

b.  Separate  users/maintainers  should  signal  separate 
CICIs. 

c.  System  performance  parameters,  such  as  MADT,  pt, 
should  be  considered  in  the  allocation  process. 

d.  "Politics"  among  the  contractors,  procuring  agency, 
using  organization,  and  maintenance  organization 
must  be  considered. 
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e.  Previous  experience  with  a  similar  type  of  system 
is  desirable. 

f.  Documentation  requirements  should  be  minimized. 

ft.  User  and  maintainer  advocacy  should  be  considered. 

h.  Size  is  no  longer  considered  a  criterion  to  be  used 
in  selecting  CPUs. 

Advanta^es/Disa dvantares  of  Tnese  Criteria.  Several 
-advantages  and  disadvantages  of  the  way  CPCIs  are  presently 
selected  were  listed  oy  the  respondents .  They  were  cate¬ 
gorized  by  tne  method  of  their  selection  (i.e.,  contractor 
chosen,  government  imposed,  previous  experience  of  the  con¬ 
tractor  and/or  SPO,  and  conformance  with  the  DoD  and  A F 
policies  and  procedures). 

In  general,  there  was  agreement  that  those  system  re¬ 
quirements  allocated  by  the  contractors  result  in  a  total 
collection  of  system  software  that  can  be  managed  in  accor¬ 
dance  with  established  Air  Force  procedures.  In  summary, 
the  respondents  described  the  following  advantages  of  con¬ 
tractor-selected  CICIsi 

a.  The  Government  can  hold  the  contractor  responsible 
for  a  totally  integrated  design  package,  which  is 
available  at  acceptance. 

b.  Cost  to  the  Government  is  minimized. 

c.  The  Government  does  not  accept  responsibility  pre¬ 
maturely. 

d.  Programmer  assignment  is  easier. 

e.  Unit  testing  and  documenting  the  system  are  easier. 
On  the  other  hand,  the  proper  application  of  CPCI  selection 
criteria  sometimes  requires  more  knowledge  and  experience 
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in  system  acquisition  concepts  and  implications  than  is 
brought  to  bear  by  the  contractors. 

At  the  other  extreme  (i.e.,  government- imposed  selec¬ 
tion  criteria/CICI  structure),  the  disadvantages  appear  to 
greatly  outweigh  the  advantages.  In  other  words,  only  three 
minor  advantages  were  mentioned  by  the  respondents?  while, 

-  several  negative  results  were  directly  attributed  to  the 
Government  direction.  For  example,  several  costly  ECFs  were 
required  on  one  program.  In  another  case,  completion  of  the 
functional  qualification  test  did  not  allow  observation  of  a 
working  portion  of  the  system. 

Although  most  of  the  respondents  cited  previous  experi¬ 
ence  as  the  primary  consideration  in  selecting  a  CFCI  struc¬ 
ture,  they  did  not  agree  on  its  utility.  For  example,  ex¬ 
perience  may  lead  to  an  unnatural  structure  because  "it  was 
done  this  way  before  or  the  people  in  charge  are  unaware  of 
any  better  technique.”  Similarly,  selection  criteria  chosen 
on  the  basis  of  experience  are  hard  to  clarify  and  often 
overlap.  This  results  in  independent  systems,  duplicated 
functions,  and  difficult  integration  problems.  On  the  other 
hand,  criteria  chosen  in  this  manner  do  minimize  the  problems 
associated  with  specification  maintenance,  testing/require¬ 
ments  traceability,  and  turnover. 

Only  2  of  the  37  respondents  indicated  that  the  CFCI 
selection  criteria  on  their  program  were  chosen  to  conform 
with  the  DoD  and  AF  policies  and  procedures.  However,  they 
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did  agree  that  the  criteria  were  understandable  and  worked 
for  most  applications,  but  that  all  configuration  contin¬ 
gencies  may  not  have  been  considered. 


Impacts  of  the  Selected  CPC  I  Structure.  The  number  and 
composition  of  configuration  items  in  the  acquisition  of  a 
major  defense  system  is  a  critical  design  issue  since  the 
Government's  technical  management  activities  primarily  fo¬ 
cuses  on  GIs  and  CICIs.  Thus,  the  need  for  a  CPCI  structure 
that  allows  management  to  draw  realistic  conclusions  about 
the  cost,  schedule,  and  performance  of  the  final  software 
system  cannot  be  overemphasized. 

Over  72 %  of  those  Government  and  industry  "software  ex¬ 
perts"  interviewed  in  this  research  reported  that  they  had 
experienced  problems  with  the  CPCI  structure  on  their  pro¬ 
gram.  In  fact,  this  is  sufficient  statistical  evidence  to 
support  the  hypothesis  that  problems  are  experienced  with 
the  CPC  I  structure  on  more  than  half  of  the  system  develop¬ 
ment/acquisition  programs  that  are  software-intensive.  An¬ 
other  interesting  statistical  inference  is  that  problems 
with  the  C1CI  structure  become  more  prevalent  as  a  program 
gets  larger. 

In  many  cases,  the  impact  of  these  problems  was  dev¬ 
astating  to  the  system  development/acquisition  program.  For 
example,  schedules  were  slipped,  documentation  became  con¬ 
fusing,  requirements  were  "lost, "  ECPs  were  generated  to 
modify  the  initial  CPCI  list,  users  became  frustrated, 


testing  requirements  were  relaxed,  costs  increased  dramati¬ 
cally,  and  CiCIs  were  delivered  with  a  large  number  of  soft¬ 
ware  discrepancies.  In  one  case,  the  impacts  were  so  ex¬ 
treme  that  the  program  was  cancelled  in  lieu  of  trying  to 
salvage  it. 

Feasibility,  Potential  Impacts,  and  Implementation  of  Hori¬ 
zontal  Allocation 

Feasibility.  Over  82>’i  of  those  responding  to  the  Soft¬ 
ware  Requirements  Decomposition  Interview  indicated  that 
horizontal  allocation  is  feasible.  In  other  words,  there  is 
sufficient  statistical  evidence  to  support  the  hypothesis 
that  more  than  half  of  the  "software  experts"  in  the  Govern¬ 
ment  and  industry  believe  that  horizontal  allocation  is  a 
feasible  methodology  for  allocating  software  requirements  to 
CICIs.  This  result  was  significantly  correlated  (positively) 
with  the  size  of  the  development/acquisition  program,  both 
in  terms  of  the  number  of  CiCIs  and  lines  of  code.  Simi¬ 
larly,  those  respondents  who  experienced  problems  with  the 
CPC  I  structure  on  their  program  were  more  likely  to  agree 
that  horizontal  allocation  is  feasible. 

Potential  Impacts.  Based  on  an  analysis  of  the  inter¬ 
view  results,  the  potential  impacts  of  defining  CPCIs  in 
•  terms  of  system  versions  or  models  (each  of  which  contains 

end-use  system  functional  capabilities)  were  overwhelmingly 
positive.  In  fact,  there  was  sufficient  statistical  evidence 
to  support  the  hypothesis  that  more  than  half  of  the 
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Government  and  contractor  "software  experts"  believe  that 
horizontal  allocation  would  be  at  least  somewhat  effective 
in  favorably  impacting  all  but  one  of  the  twelve  evaluation 
parameters.  In  addition,  over  60%  of  the  respondents  indi¬ 
cated  that  this  approach  to  CiCI  selection  would  be  moder¬ 
ately  to  very  effective  in: 

a.  providing  the  program  manager  and  user  with  more 
objective  visibility  into  the  system  development, 

b.  improving  the  software  quality, 

c.  reducing  the  software  integration  task,  and 

d.  highlighting  problems  earlier. 

The  bivariate  correlations  among  the  12  evaluation  pa¬ 
rameters,  and  between  these  parameters  and  the  demographics/ 
program  characteristics  produced  several  significant  inter¬ 
dependencies.  For  example,  the  respondents'  opinions  on  the 
effectiveness  of  horizontal  allocation  in  making  debugging 
and  testing  more  efficient,  making  software  maintenance 
easier  and  less  costly,  increasing  the  morale  of  those  work¬ 
ing  on  the  program,  reducing  the  complexity  of  the  decompo¬ 
sition  process,  and  improving  training  effectiveness  were  all 
positively  correlated  with  the  number  of  CICIs  on  those  pro¬ 
grams  evaluated. 

Implementation.  Over  75%  of  those  participating  in  the 
survey  indicated  that  horizontal  allocation  should  be  im¬ 
plemented  on  future  software  development/acquisition  pro¬ 
grams.  This  result  was  significantly  correlated  (positively) 
with  the  size  of  the  program  evaluated,  both  in  terms  of  the 
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number  of  CICIs  and  lines  of  code.  In  other  words,  respon¬ 
dents  who  evaluated  large  developrr.ent/acquisition  programs 
believed  that  this  approach  should  be  implemented  in  the 
future . 

Conclusions 

The  importance  of  computer  hardware  and  software  in  the 
Department  of  Defense  has  increased  over  the  past  20  years 
to  the  point  where  computer  technology  is  vital  to  the  de¬ 
fense  of  our  country.  In  addition,  tnis  technology  has 
placed  a  tremendous  strain  on  the  fiscal  assets  of  the  DoD 
and  the  AF.  Therefore,  the  need  to  manage  computer  hardware 
and  software  as  critical  components  of  a  defense  system 
throughout  its  life  cycle  is  becoming  widely  recognized.  A 
general  awareness  of  this  need  was  reflected  in  the  survey 
of  Government  and  industry  "software  experts,"  as  well  as  in 
the  literature. 

Although  the  selection  of  CICIs  is  one  of  the  most 
critical  decisions  made  during  the  acquisition  of  a  software¬ 
intensive  system,  no  known  and  proven  approach  was  uncovered 
in  this  research.  On  most  programs  evaluated,  functional 
modularity  and  previous  experience  were  the  primary  consid¬ 
erations  used  by  SPOs  and  contractors  to  allocate  system  re¬ 
quirements  into  CPCIs.  This  resulted  in  many  problems  with 
assessing  development  status,  achieving  system  integration, 
completing  meaningful  tests  and,  documenting  the  system.  In 
essence,  managing  a  program  partitioned  in  this  manner  forces 
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the  program  manager  to  make  an  inductive  evaluation  that  the 
system  will  ultimately  work  if  and  only  if  all  other  CPCIs 
work. 

On  the  other  hand,  selecting  CiCIs  on  the  basis  of  hor¬ 
izontal  allocation  promises  to  favorably  impact  the  cost, 
schedule,  performance,  and  management  parameters  normally 
associated  with  an  acquisition  program.  This  alternate  tech¬ 
nique  is  based  on  defining  a  software-intensive  system  in 
terms  of  system  versions  or  models,  each  of  which  contains 
end-use  system  functional  capabilities.  Therefore,  a  CPC  I 
is  equated  to  a  version/model  or  series  of  versions/models 
for  management  purposes.  In  other  words,  a  CPCI  is  a  func¬ 
tional  path  or  sequence  of  activities  that  results  in  the 
production  of  a  required  response  from  a  set  of  available 
inputs  or  stimuli. 

This  alternative  technique  for  allocating  system  re¬ 
quirements  to  CPCIs  allows  a  system  to  be  developed  on  an 
incremental  basis  rather  than  the  traditional  "black-box" 
approach.  In  other  words,  incremental  development  provides 
a  system  capability  that  is  operational,  rather  than  pieces 
and  parts  of  a  system  that  are  only  partially  operational. 
Thus,  "builds"  represent  functi >nal  capabilities  and  become 
the  management  milestones  of  system  development. 

Recommendations 

The  recommendations  that  follow  are  intended  to  provide 
suggested  ways  of  improving  the  process  of  developing/ 
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acquiring  software  embedded  in  major  defense  systems. 

a.  A  total  systems  approach  to  the  CiCI  selection  pro¬ 
cess  is  needed.  To  do  this,  it  is  necessary  to  con¬ 
sider  the  following  factors/questions  prior  to  fi¬ 
nalizing  the  CICI  structure: 

(1)  Advantages  and  disadvantages  of  each  alterna¬ 
tive  CICI  structure. 

(2)  Roles  and  responsibilities  of  each,  partici¬ 
pating  organization  (i.e.,  user,  developer, 
maintainer,  and  procurer). 

(3)  Level  of  management  visibility  required. 

(4)  Previous  experience  with  a  similar  type  of 
system. 

(5)  Installation  location  and  functions  to  be  per¬ 
formed. 

(6)  Contractual  and  legal  implications  of  a  CiCI. 

b.  The  confusion  ar.d  conflict  caused  by  the  ASPR  and 
AFSCP  800-3  treating  computer  software  as  an  item 
of  data,  rat.ner  than  an  active  system  component, 
should  be  terminated  by  changing  these  directives. 

c.  The  discrepancies  in  AFR  800-14  and  MIL-3TD-483  re¬ 
garding  definition  of  the  allocated  baseline  need 
to  be  resolved. 

d.  Since  size  is  no  longer  considered  a  criterion  to 
be  used  in  selecting  CPC  Is,  it  should  be  deleted 
from  all  lists  of  CICI  selection  criteria  (Ref. 

AFSCP  800-?,  1977:76  and  Glore,  1977:27). 

e.  The  AFR  65-3  definition  of  a  Cl  (CPCI  is  undefined 
in  the  regulation)  should  be  clarified  by  changing 
"end-use  function"  to  "end-use  system  capability," 
The  rationale  for  this  is  that  the  definition  as 
written  leads  to  "vertical"  allocation  of  software 
requirements  to  CICIs.  This  structure  typifies  the 
"black  box"  approach;  and  therefore,  creates  arti¬ 
ficial  interfaces. 

f.  HQ  AFSC  should  fund  the  development  of  an  acquisi¬ 
tion  management  guidebook  that  addresses,  in  detail, 
the  system  requirements  allocation  process  (for  CIs 
and  CPC  Is).  A  comprehensive  example,  as  well  as  the 
results  of  this  research  should  be  included.  In  the 


meantime,  the  results  of  this  research  should  be 
published  and  made  available  to  the  software  in¬ 
dustry,  as  well  as  those  in  the  Government  (pro¬ 
curers,  developers,  users,  and  maintainers ) . 

g.  Horizontal  allocation  should  oe  implemented  on  a 
medium  to  small  sized  program  to  establish  empiri¬ 
cal  evidence  that  it  is  as  effective  as  the  "soft¬ 
ware  experts"  believe  it  would  be. 

h.  Since  the  number  of  "software  experts"  interviewed 
was  limited,  the  survey  should  be  expanded  to  in¬ 
clude  a  larger  number  of  respondents  and  variety  of 
programs.  This  would  increase  confidence  in  the 
results  achieved  and  conclusions  derived  in  this 
study. 
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APPENDIX  A 


GLOSSARY  OP  TERMS 


Acqui sit  ion.  The  act  of  gaini ng  something  through  the  con¬ 
ceptual,  validation,  full-scale  development,  production  and 
deployment  phases  of  the  system  life  cycle. 

Acquisition  Life  Cycle.  The  path,  divided  into  phases, 
through  which  a  program  progresses  between  initial  identi¬ 
fication/documentation  of  an  operational  requirement  and 
finally  retirement  from  the  operational  retirement  (AFSCP 
800-7,  1977*84). 

Allocation  of  Requirements.  The  act  of  apportioning  the 
system  performance  and  functional  requirements  to  individ¬ 
ually-identified  subsets  for  purposes  of  managing  their  de¬ 
velopment.  The  identification  of  CiCIs  is  the  typical  re¬ 
sult  of  this  apportioning  process. 

Basel ine .  The  documented  requirements  of  a  system  that  are 
implemented  through  multilateral  agreements  among  a  developer, 
procurer,  and  user  (AFSCP  800-3,  1976:9-3). 

Build.  A  collection  of  partial  or  complete  program  modules 
and  data  which,  operating  together,  performs  selected  func¬ 
tions. 

Computer  Equipment  (or  Computer  Hardware).  Devices  capable 
of  accepting  and  storing  computer  data,  executing  a  system¬ 
atic  sequence  of  operations  on  computer  data  or  producing 
control  outputs  (DoDD  5000,29,  1976:i£ncl  l). 

Comruter  Irorram.  A  series  of  instructions  or  statements  in 
a  form  acceptable  to  computer  equipment,  designed  to  cause 
the  execution  of  an  operation  or  series  of  operations.  Com¬ 
puter  programs  include  such  items  as  operating  systems,  as¬ 
semblers,  compilers,  interpreters,  data  management  systems, 
utility  programs,  maintenance/diagnostic  programs  and  ap¬ 
plications  programs  (DoDD  5000.29,  1976:Encl  l). 

Comruter  Resources.  The  totality  of  computer  equipment, 
computer  programs,  computer  data,  associated  documentation, 
personnel,  and  supplies.  These  resources  must  be  managed 
as  elements  or  subsystems  of  major  importance  during  all 
phases  of  the  system  acquisition  life  cycle  (DoD  5000.29, 
1976iEncl  1). 

Computer  Softv.-are.  A  combination  of  associated  computer 
programs  and  computer  data  required  to  enable  the  computer 
equijment  to  perform  computational  or  control  function  (DoDD 
5000.29,  1976 : Enel  1). 
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Gonf i, duration  Identif ication  (or  Cl/GiG I  Selection).  The 
process  of  documenting  the  performance,  qualification,  fab¬ 
rication,  and  acceptance  requirements  of  a  system  (AFSCF 
800-7,  1 977  » 1 6 ) . 

Configuration  Item.  An  aggregation  of  hardware/computer 
programs  or  any  of  its  discrete  portions,  which  satisfies 
an  end-use  function  and  is  designated  by  the  Government  for 
conf iguration  management  (AFR  65-3,  197^:A-2). 

Gonf ir-uration  Management.  The  management  discipline  whose 
purpose  is  to  identify,  control,  account  for,  and  audit  the 
functional  and  physical  characteristics  of  systems,  equip¬ 
ments,  and  other  designated  items  developed,  produced, 
operated,  and  supported  by  D oD  components  (AFR  65~3»  1975 « 
1-1). 


Data.  In  this  thesis,  data  refers  to  reports,  forms,  manuals, 
specifications,  and  other  items  of  the  classes  which  are 
acauired  via  a  Contract  Data  Requirements  List  (Learie, 
1977:111). 

Decomposition  of  Requirements.  The  act  of  hierarchially 
separating  system  or  subsystem  requirements  into  units  of 
development,  such  as  functions,  tasks,  modules,  subroutines, 
components,  and  instructions. 

Embedded  Computer  Gy stems.  A  special  subset  of  computer 
applications  that  are  physically  incorporated  into  a  larger 
system.  Their  primary  function  is  not  data  processing  and 
their  outputs  generally  include  control  signals. 

Functional  Modularity.  The  process  of  partitioning  a  system 
into  modules,  or  standardized  units,  on  the  basis  of  group¬ 
ing  like  functions  to  be  performed  by  the  system. 

Horizontal  Allocation.  An  approach  to  allocating  system  re¬ 
quirements  to  GlCIs  on  tne  basis  of  system  versions  or 
models,  each  of  which  contains  end-use  system  functional 
capaoilities.  A  system  defined  in  this  manner  would  be  de¬ 
veloped  on  an  incremental  basis  in  whicn  each  "Duild"  pro¬ 
vides  a  system  capability  tnat  is  operational,  rather  than 
pieces  and  parts  that  are  only  partially  operational. 

Major  System  Acquisition.  A  designation  made  by  the 
Secretary  of  Defense.  System  programs  involving  an  antici¬ 
pated  cost  of  $75  million  in  research,  development,  test  and 
evaluation  or  $300  million  in  production  are  considered  for 
designation  as  major  system  acquisitions  (DoDD  5000,1,  1977 » 
2). 
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Iromram  Manager.  The  person  tasked  to  plan,  organize,  and 
conduct  the  acquisition  of  a  system,  subsystem  or  component 
within  apnroved  limits  of  system  performance,  and  funding 
(AFR  800-2,  1977:1-2). 

Software  Fnmineering.  Science  of  design,  development,  im¬ 
plementation,  test,  evaluation,  and  maintenance  of  computer 
software  over  its  life  cycle  (DoDD  5000. 29»  1976:i£ncl  1). 

Software-Intensive  System.  A  system  that  uses  a  high  con¬ 
centration  of  software  to  perform  its  intended  function. 

System.  A  composite  of  items,  assemblies  (or  sets),  skills, 
and  techniques  capable  of  performing  and/or  supporting  an 
operational  (or  non-operational  role)  (AFR  65-3,  1974 :A-3). 

System  Version  (or  Model).  The  actual  configuration  of  a 
computer  program  configuration  item(ClCI)  which  is  intro¬ 
duced  for  installation  and  test  or  operation  into  the  system 
in  the  form  of  a  magnetic  tape,  deck  of  cards,  disc,  or  other 
physical  medium. 

Weapon  System.  Any  DoD  system  that  is  not  of  a  general  pur¬ 
pose,  commercially  available  nature.  It  includes  aircraft 
systems;  space  and  missile  systems j  command,  control,  and 
communications (c3)  systems;  and  intelligence  systems. 
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LIST  OF  ACRONYMS  AND  ABBREVIATIONS 


* 


* 


* 


AC  I  Allocated  Configuration  Identification 

Acq.  Acquisition 

ADP  Automatic  Data  Processing 

ADPE  Automatic  Data  Processing  Equipment 

AF  Air  Force 

AFB  Air  Force  Base 

AFIT  Air  Force  Institute  of  Technology 

AFLC  Air  Force  Logistics  Command 

AFR  Air  Force  Regulation 

AF3C  Air  Force  Systems  Command 

AF3CP  Air  Force  Systems  Command  Pamphlet 

AF3CRP  Air  Force  Systems  Command  Recurring  Pamphlet 

ASD  Aeronautical  Systems  Division 

ASFR  Armed  Services  Procurement  Regulations 

ATC  Air  Training  Command 


BMDATC  Ballistic  Missile  Defense  Advanced  Technology 

Center 

C-^  Command,  Control,  and  Communications 

CKI  Command  and  Control 

CADSAT  Computer  Aided  Design  and  System  Analysis  Tool 

Capt.  Captain 

CDC  Control  Data  Corporation 

CDR  Critical  Design  Review 

CDRL  Contract  Data  Requirements  List 

Cl  Configuration  Item 

Col.  Colonel 

Cont.  Continued 

CPC  Computer  Program  Component 

CPC  I  Computer  Program  Conf iguration  Item 
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DCP 

Dev. 

d. f. 

DoD 

DoDD 

DoDI 

DPG 

DSARC 

DSMC 

ECP 

ECS 

e. g. 

ESD 
et  al 
etc. 

F3 

FCA 

FCI 

FCRC 

FQT 

GFE 

GSA 

HQ 

ID 

i.e. 

Inc . 

JLC 

Lt.  Col. 


Decision  Coordinating  Paper 

Development 

Decrees  of  Freedom 

Department  of  Defense 

Department  of  Defense  Directive 

Department  of  Defense  Instruction 

Defense  Procurement  Circular 

Defense  System  Acquisition  Review  Council 

Defense  Systems  Management  College 

Engineering  Change  Proposal 
Embedded  Computer  System 
for  example 

Electronic  Systems  Division 
and  others 
et  cetera 

Form,  Fit,  and  Function 
Functional  Configuration  Audit 
Functional  Configuration  Identification 
Federal  Contract  Research  Center 
Functional  Qualification  Test 

Government  Furnished  Equipment 
General  Services  Administration 

Headquarters 

Identification 
that  is 
Incorporated 

Joint  Logistics  Commanders 
Lieutenant  Colonel 
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MADT  Maximum  Allowable  Down-time 

Maj.  Major 

MENS  Mission  Element  Need  Statement 

MIL-STD  Military  Standard 

MIL-SPEC  Military  Specification 

N/A  Not  Applicable 

NBS  National  Bureau  of  Standards 

OMB  Office  of  Management  and  Budget 


P.L. 

PM 


Physical  Configuration  Audit 

Product  Configuration  Identification 

Preliminary  Design  Review 

Public  Law 

Program  Manager 

Program  Management  Directive 

Program  Management  Plan 

Program  Office 

Processing  Time 


RADC 


RDT&E 


Rome  Air  Development  Center 
Research  and  Development 

Research,  Development,  Test  and  Evaluation 


SADT 


SECDEF 


SPECS 


SPSS 


Structured  Analysis  Design  Technique 

Secretary  of  the  Air  Force 

System  Development  Corporation 

System  Design  Review 

Secretary  of  Defense 

System  Manager 

Statement  of  Work 

Specifications 

System  Program  Office 

Statistical  Package  for  the  Social  Sciences 
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Mr* 


SRWG 

SYS 


Software  Reliability  Working  Group 
Systems 


Tech. 

Technical 

TR 

Technical  Report 

USAP 

United  States  Air  Force 

% 

Vol. 

Volume 

4 

VP 

Vice-President 

0 

WWMCCS 

World  Wide  Military  Command  and  Control  System 

4 


1 


* 


* 
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APPENDIX  C 


ire  Cover  Letters  f< 


and  Personal  Interv 


Rinv  to 

ATTN  Of  ENA 


DEPARTMENT  OF  THE  AIR  FORCE 
AIR  FORCE  INSTITUTE  OF  TECHNOLOGY  <  ATC  > 
WRIGHT-PATTERSON  AIR  FORCE  BASE.  OHIO  45433 


•u.j.ct  Master's  Degree  Thesis  Interview 


10  May  ?9 


TO. 


1.  A3  promised  in  our  recent  phone  conversation,  a  brief 
description  of  the  Software  Requirements  Decomposition 
Interview  is  enclosed  for  your  information.  The  specific 
purpose  of  the  study  and  format  of  the  interview  instrument 
are  described.  In  addition,  a  request  for  your  cooperation 
in  completing  this  vital  data  collection  effort  is  included. 

2.  Based  on  pretest  analyses,  the  interview  should  take 
approximately  45  minutes  to  complete.  The  specific  date 
and  time  of  the  interview  will  be  arranged  with  your  office 
in  the  next  3-4  weeks. 


3.  Questions  or  comments  regarding  the  interview  or  the 
thesis  topic  should  be  addressed  to  me  at  the  above  address 
or  telephone  number  51 3-255-5533  (Autovon  785-5533)* 


rir\ 


Virgil  L.  Cooper,  Capt,  USAF 
Graduate  Student,  GSK-79S 
School  of  Engineering 


1  Atch 

Software  Requirements 
Decomposition  Intarview 
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SOFTWARE  REQUIREMENTS  DECOMPOSITION  INTERVIEW 


Purpose  of  the  Study 

The  objective  of  this  study  is  to  investigate  one  por¬ 
tion  of  the  process  of  decomposing  system  performance  re¬ 
quirements  into  individually-identified  subsets  for  purposes 
of  managing  their  development.  These  subsets,  called  con¬ 
figuration  items (CIs)  for  hardware  and  computer  program 
conf iguration  items(CICIs)  for  software,  are  normally 
identified  during  the  early  stages  of  a  system  acquisition 
program.  This  process  is  guided  by  the  Air  Force  policy 
that  there  must  be  a  recognized  and  documented  initial 
statement  of  requirements,  and  that  once  stated,  any  change 
in  requirements  will  be  documented  so  that  the  current 
status  of  a  program  is  known  (AFJCP  800-7). 

CIs  and  CPC  Is  are,  therefore,  regarded  as  the  level  of 
management  at  which  a  program  office  exercises  formal  man¬ 
agement  control.  Specifically,  they  are  the  leveli 

a.  At  which  the  procuring  activity  specifies,  con¬ 
tracts  for,  and  accepts  individual  parts  of  the  system. 

b.  Jielow  which  the  developer  is  responsible  for  man¬ 
agement  of  the  development  or  procurement,  and  assembly  of 
item  components. 

c.  Above  which  the  procuring  activity  retains  the  re¬ 
sponsibility  foi  delivering  a  complete  system  that  meets 
the  specified  ^stem  requirements. 

The  portion  of  the  decomposition  process  that  is  of 
concern  in  this  research  effort  is  the  selection  of  CPCIs 
on  a  software-intensive  system.  Normally,  the  performance 
and  functio  j.1  requirements  of  a  system  are  documented  in 
a  system  o?  system  segment  specification.  These  requirements 
are  then  allocated  to  CIs  and  CPCIs  based  on  selection 
criteria  defined  by  the  procuring  activity  and/or  the  de¬ 
velopment  contractor.  The  specific  topic  of  this  interview, 
then,  is  the  criteria  used  for  decomposing  system  require¬ 
ments  to  be  implemented  via  software  into  CPCIs. 

The  research  data  gathered  in  this  study  will  support 
a  master's  degree  thesis  at  the  Air  Force  Institute  of  Tech¬ 
nology  (AFIT) .  This  topic  was  chosen  for  my  research  project 
for  two  reasons.  First,  I  experienced  many  of  the  frustra¬ 
tions  and  disappointments  associated  with  software  acquisi¬ 
tion  as  a  member  of  a  major  command  and  control  systems 
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program  office  in  my  last  assignment.  During  that  four 
year  period,  I  participated  in  all  phases  of  the  system 
acquisition  cycle  as  a  Computer  Systems  Analyst  and  a 
System  Integration  and  Test  Monitor.  Secondly,  my  projected 
follow-on  assignment  is  to  a  Headquarters  Air  Force  Ac¬ 
quisition  Study  Group.  Hopefully,  the  lessons  learned  in 
my  previous  assignment  and  the  results  of  this  research  will 
contribute  to  improvements  in  the  Air  Force  weapons  system 
acquisition  procedures. 

You  have  been  selected  as  part  of  a  sample  of  govern¬ 
ment  and  contractor  personnel  who  are  presently (or  have  been 
in  the  recent  past)  responsible  for  one  of  the  various  dis¬ 
ciplines  of  software  development  and  acquisition  management. 
Your  cooperation  is,  therefore,  sincerely  requested.  An¬ 
swers  or  comments  you  provide  may  be  included  in  published 
articles,  reports,  or  texts.  However,  an  individual  and 
his  organization  will  not  be  identified  with  his  comments. 

Interview  Structure 


This  interview  was  designed  to  capture  both  objective 
and  subjective  data  on  previous  CFCI  selection  efforts,  as 
well  as  the  respondents '  opinions  on  an  alternate  technique 
of  decomposing  the  system  requirements.  The  questions  are 
divided  into  three  sections.  Section  I  involves  demographic 
questions,  such  as  organizational  level,  type  of  job,  ex¬ 
perience  level,  size  of  project  under  development,  etc. 
Section  II  contains  questions  on  the  criteria  used  for  de¬ 
composing  system  requirements  on  recent  and  existing  de¬ 
velopmental  projects.  In  addition,  the  advantages  and  dis¬ 
advantages  of  the  criteria  are  of  interest.  Section  III 
involves  questions  on  the  feasibility  and  impact  of  an 
alternate  technique  for  selecting  CFCls.  Specifically, 
data  will  be  collected  on  the  perceived  impacts  on  the 
system  cost,  system  development  time,  system  performance, 
management  visibility,  and  contractor's  management  pro¬ 
cedures. 


DEPARTMENT  OF  THE  AIR  FORCE 
AIR  FORCE  INSTITUTE  OF  TECHNOLOGY  ( ATC ) 
WRIGHT- PATTERSON  AIR  FORCE  BASE.  OHIO  45433 


ENA  4  Jun  79 

Software  Requirements  Decomposition  Interview 


1.  After  completing  some  of  the  subject  interviews  at  ASD  and 
HQ  AFLC,  I  concluded  that  some  of  the  questions  may  require 
some  considerable  thought  on  your  part.  Therefore,  I  have 
enclosed  the  questionaire  and  an  example  referred  to  in  Section 
II  of  the  questionaire.  If  you  have  time  prior  to  my  arrival, 
please  answer  all  questions  that  don't  require  any  further 
clarification.  This  procedure  will  require  less  time  for  our 
meeting  and  will  be  more  beneficial  to  my  research. 

2.  I  will  be  contacting  your  office  this  week  to  arrange  the 
specific  time  for  our  meeting. 

3.  Questions  or  comments  should  be  addressed  to  me  or  the  above 
address  or  at  commercial  513-236-2944  (Autovon  785-5533/3030). 


— • 

VIRGIL  L.  COOPERf  Captain,  USAF  1  Atch 

Graduate  Student,  GSM  79S  Questionnaire  w/example 

School  of  Engineering 
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Strength  Through  Knowledge 


SOFTWARE  REQUIREMENTS  DECOMPOSITION  INTERVIEW 


Capt.  Virgil  Lee  Cooper 
Graduate  Student,  GSM-79S 
School  of  Engineering 
Air  Force  Institute  of  Technology 
Wright-Patterson  Air  Force  Base,  Ohio 


Interview  Method 


Date/Time 


Respondent 


Phone  Number 


Privacy  Statement 

In  accordance  with  paragraph  30,  AFR  12-35*  Air  Force 
Privacy  Program,  the  following  information  about  this  inter¬ 
view  is  provided! 

a.  Authority. 

(1)  4  U.S.C.  301*  Departmental  Regulations  and/or 

(2)  10  U.S.C.  8012,  Secretary  of  the  Air  Force. 

Powers  and  Duties.  Delegation  by. 

b.  Principal  purposes.  This  interview  is  being  con¬ 
ducted  to  collect  information  to  be  used  in  research  aimeu 

at  illuminating  and  providing  inputs  to  the  solution  of  prob¬ 
lems  of  interest  to  the  Air  Force  and/or  DOD. 

c.  Routine  Uses.  The  interview  data  will  be  converted 
to  information  for  use  in  research  of  system  acquisition  re¬ 
lated  problems.  Results  of  the  research  will  be  included  in 
a  written  master’s  thesis  and  may  also  be  included  in  pub¬ 
lished  articles,  reports,  or  texts.  Distribution  of  the  re¬ 
sults  of  the  research,  whether  in  written  form  or  orally 
presented,  will  be  unlimited. 

d.  Participation  in  this  interview  is  entirely  voluntary. 

e.  No  adverse  action  of  any  kind  may  be  taken  against 
an  individual  who  elects  to  participate  in  any  or  all  of 
this  interview. 
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What  is  your  organizational  level  of  assignment? 


_  Air  Staff 

_  KAJCOM  Staff 

_  Product  Division  Staff (AFSC) 

_  Systems  Program  Office(SPO) 

_  Other.  Please  specify  _ 

2.  What  function  are  you  assigned  to? 

_  Command  Section 

_  Program  Management 

_  Program  Control 

_  Engineering:  _  Software 

_  Hardware 

_  System 

_  Test  and  Evaluation 

_  Configuration  Management 

_  Logistics 

_  Data  Management 

_  Other.  Please  specify  _ 


3.  What  is  your  present  grade? 

4.  What  is  your  highest  level  of 
Education? 
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5.  Have  you  ever  had  any  formal  training  (e.g.,  College  Courses, 
Technical  Courses,  etc)  ini 

a.  System  Engineering?  _ 

b.  Software  Engineering?  _ 

c.  Hardware  Engineering?  _ 

d.  Acquisition  Management?  _ 


6.  How  many  years  have  you  been 
involved  in  systems  develop¬ 
ment/acquisition?  _  years 


7.  How  many  of  those  years  have 
you  been  involved  in  the  de¬ 
velopment/acquisition  of  com¬ 
puter  software?  _  years 


8,  How  recent  was  the  experience? 


9.  What  was  the  primary  application  of  the  computer  software? 

_  Airborne  Systems 

_  Intelligence 

_  Command,  Control,  and  Communication 

_  Space  and  Missile 

_  Scientific 


Business 


Other.  Please  specify 


10.  What  type  of  software  experi¬ 
ence  do  you  have  (e.g.,  pro¬ 
gramming,  systems  analysis, 
maintenance,  test  and  eval¬ 
uation,  etc)? 


11.  How  many  system  acquisition 
programs  have  you  been  in¬ 
volved  with? 
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Process 


’1 


Section  1 1 »  Present  CPC  I  Selection 


1 .  What  is  the  Program  ID  of  the 
program  you  are  presently (or 
were  last)  associated  with? 


2.  Briefly  describe  the  program  (e.g.,  what  is(was)  being  de¬ 
veloped?  How?  Major  program?  etc). 


3.  How  long  have  you  been (were  you) 

assigned  to  the  program?  _  years 

4.  Briefly  describe  your  job  on  the  program. 


At  what  point  in  the  system 
acquisition  cycle  were  the 
system  requirements  allocated 
to  CPCIs  (e.g.,  conceptual 
phase,  first  part  of  full- 
scale  development  phase,  etc)? 


Who  performed  the  functions  of 
CPCI  selection  (e.g.,  contractor 
Government  software  engineers, 
etc)? 


* 


?.  What  are  the  characteristics  of  the  program  you  are  presently 
(were)  associated  with  in  terms  ofi 


a.  Total  development  cost? 

b.  Number  of  CFCIs? 

c.  Number  of  lines  of  code 
(i.e.,  executable  in¬ 
structions)? 

d.  Cost  ratio  of  software  to 
hardware  ? 

e.  Percentage  of  software 
development/contract 
period  that  is  complete? 


8.  Do  you  agree  that  the  CPC  Is  on 
your  present (past)  program  were 
all  defined  at  about  the  right 
size  in  terms  of  managing  their 
development? 

If  no,  please  explain. 


9.  Are  you  familiar  with  the  cri- 
*  teria  used  on  your  present(past) 

program  for  allocating  the 
system  requirements  to  CPC  Is? 

If  so,  can  you  identify  some  of 
them? 
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10.  How  were  these  criteria  selected? 


11.  What  are  the  advantages  and  disadvantages  of  these  criteria? 


12.  Are  these  criteria  different 

from  those  used  on  any  previous 
programs  you  have  been  associated 
with? 

If  so,  what  were  they? 


195 


13.  What  are  the  advantages  and  disadvantages  of  those  criteria 


14.  In  retrospect,  can  you  identify 
any  other  criteria  that  should 
have  been  used  in  decomposing 
the  system  requirements? 

If  so,  what  are  they  and  why 
should  they  have  been  used? 


15.  Do  you  believe  that  the  criteria 
used  for  selecting  CPCIs  are 
different  from  those  used  for 
decomposing  system  requirements 
into  configuration  items (CIs)? 
Explain. 
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16. 


Have  you  experienced  any  problems 
with  the  CPC  I  structure  on  your 
present (past)  programs? 

If  so,  how  were  they  solved 
and  what  was  the  impact? 


17.  Do  you  believe  that  an  adequate 
analysis  of  the  alternative  CPCI 
structures  was  performed  on  your 
present (past)  program? 

If  not,  what  should  have  re¬ 
ceived  more  emphasis? 


18.  What  changes,  if  any,  would  you  like  to  see  in  the  way 

CPC Is  are  selected  (e.g.,  in  philosophy,  analysis  techniques, 
regulations,  procedures,  etc)?  Why? 
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Section  III >  Alternate  CPC  I  Selection  Technique 


The  questions  in  this  section  are  designed  to  investigate 
the  feasibility  and  evaluate  the  potential  impacts  of  an  alter¬ 
nate  technique  for  decomposing  system  requirements  into  CPCIs. 
Specifically,  the  alternate  technique,  called  "horizontal  al¬ 
location,"  is  based  on  defining  a  software-intensive  system  in 
terms  of  system  versions  or  models,  each  of  which  contains  end- 
use  system  functional  capabilities  (See  Example).  For  manage¬ 
ment  purposes,  each  version  or  model  would  be  defined  as  a  CPC  I 
In  its  simplest  terms  then,  a  CPCI  is  a  functional  path  or  se¬ 
quence  of  activities  that  results  in  the  production  of  a  re¬ 
quired  response  from  a  set  of  available  inputs  or  stimuli. 
Therefore,  a  complete  set  of  CPCIs  and  CIs  would  satisfy  all 
the  functional  and  performance  requirements  of  the  system.  A 
system  defined  in  this  manner  would  be  developed  on  an  incre¬ 
mental  basis  in  which  each  "build"  provides  a  system  capability 
that  is  totally  operational,  rather  than  pieces  and  parts  that 
are  only  partially  operational.  Thus,  "builds"  represent  func¬ 
tional  capabilities  and  become  the  milestones  of  system  develop 
ment. 


Improving  the  software 
quality  (e.g.,  fewer  latent 


How  effective  do  you  believe  that  it  would  be  im  (cont. 


a 


•H 

>>P 
P  O 
o >  a> 

>  <M 
<M 

tu 


>> 

P  0) 
0)  > 
P  *H 
rt  +-> 
P  o 

d)  0) 
TJ  «M 

O  <m 
22  cq 


d) 

P  > 
ctJ  p 
X  p 
5  o 

<U  (U 
E  <M 
O  <M 
to  Ci) 


0) 
> 
•H 
p p 

o  o 
a  d) 
«m 

<M 

w 


hi) 

O  1 

c 

P  P  CO 

•H 

OOP 

■H 

W 

O 

CD 

X  >  Vh 

-P 

POO 

•h  -a 

TJ 

i  a> 

C 

M 

d)  o 

P  p 

••  w  M 

P  o 

hC  C 

ID  CP 

hi  o 

X  p  p 

3  P 

p  d) 

XI  o 

>1  p 

d>  p 

hOP 

TJ  Cm 

K  P  TJ 

«m 

j  p  c  e- 

to  <u 

•o  p  (TJ  6 

c 

•mx  a> 

•H  <D 

>p  p  p 

O  CO  c  CO 

cC  O 

2*  E 

P  p  d>  >> 

Pi  >  £  CO 

-TJ 

1 

•  c 

C 

0) 

W)  (TJ 

•M 

o 

• 

(TJ 

c 

Q>  CD 

E 

oJ 

^  6 

c 

O 

d> 

>>p  p 

p 

rl 

C  P  ho  P 

•  M 

CO  c 

O 

(TJ 

O  *M 

•M 

E 

O  P 

CO 

o 

ccS 

O 

CO  o 

0) 

P 

to  X 

CTj 

d)  CO 

c 

* 

p  d) 

o 

P 

p 

p 

«M  TJ  X 

p 

o 

C  3 

(TJ 

CO 

«J  o 

p 

p 

c 

WPP 

d)  C" 

c 

d> 

E 

•H 

P  CO 

3  C 

M 

CO  CO 

O  P 

(0 

nJ  d> 

o  (TJ 

sg 

dlPTJP 

i 

I  c 

O  d) 
p 

(X  d) 

(4 
*M  CTJ 

03 

0)  *M 

P  O 
cd  co 

(4 

o  * 

S  W 
p 
d>  d) 
X  TJ 
P  o 
o 
M 

c  - 

•M  CO 
CO  P 
rt  d) 

a)  e 


p 

o 


I 

c 

o 

CO 

p 

<D 

Pi 

P 

c 

<u 

E 

<u 

hf> 

ro 

c 

cc! 

6 

TJ 

C 

rt 


CO 

P 

d> 

<u  o- 
C  P 
H  d> 

h£  C 


O 


hi) 


CM 


20J 


How  effective  do  you  believe  that  it  would  be  iru  (cont 


-*->  nc 

CJ  HO  C 
0)  C  -H 
-H  C 

X)  -H 

ci-h  m 


rH 

Hi  o  +>  o- 

P-  c 

CP  -v 

6  O 

•H  Pi  c  p 

O  -H 

C  o  o 

O  +•> 

•H  >,  |  +> 

•H 

rt  X>  W  R) 

a)  w 

P  T)  P 

x:  o 

+>  *  C  <D 

4->  a, 

•  tf  P> 

£ 

Hi  Hix:  o 

Wi  O 

c  • 

C  o 

•H  Q)  <D 

•H  <D 

>  ^  a>  -C 

O  TJ 

O  -H  -(-> 

3 

P  M  rH 

•o  o) 

Pi  w  p  p 

a>  sz 

e  <D  rt  O 

az  +» 

M  C  0)  <(H 

3. 


'1 


4. 


Do  you  think  this  technique  would 
adversely  impact  the  contractor’s/ 
developer's  management  procedures? 
Explain. 


Can  you  think  of  any  other  ad¬ 
vantages  or  disadvantages  of  this 
technique? 

If  so,  what  are  they? 


Example  of  Simple  Snacetrack  System 


The  purpose  of  this  system  is  to  maintain  a  catalog  of  all 
man-made  ofjects  orbiting  the  earth  and  to  provide  specific  statu 
and  management  reports.  The  system  is  capable  of  receiving  and 
processing  messages  from  sensor  radars,  cameras,  and  visual  sight 
infs.  Using  these  inputs,  the  system  determines  orbital  param¬ 
eters,  correlates  data  with  known  objects,  predicts  future  posi¬ 
tions,  detects  changes  in  orbit,  predicts  satellite  reentry  and 
impact,  and  identifies  and  catalogs  new  objects  launched  into 
space.  During  this  processing  and  at  the  conclusion  of  specific 
functions,  the  system  provides  information  (e.g.,  alarms,  visual 
displays  or  hard  copy  reports)  to  operational  personnel  upon 
which  they  will  base  decisions  for  controlling  the  operation  of 
the  system. 
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APPENDIX  E 


Table  of  Personnel  Responding  to  Interview 


ive  Organ 


TABLE  III 


PERSONNEL  RESPONDING  TO  INTERVIEW  QUESTIONNAIRE 


Dale  Albericci,  Hughes  Aircraft  Company,  Fullerton, 

California 

Phil  Anderson,  Air  Force  Systems  Command  Headquarters, 

Andrews  AFB,  Maryland 

Dean  Bergstrom,  Rome  Air  Development  Center  (AFSC),  Griff iss 

*  AFB,  New  York 

Bob  Berri,  Aerospace  Corporation,  Los  Angeles,  California 

.  Bill  Bjerstadt,  MITRE  Corporation,  Colorado  Springs,  Colorado 

.  Mike  Bloom,  MITRE  Corporation,  Bedford,  Massachusetts 

Charlie  Bobbish,  Electronic  Systems  Division,  Hanscom  AFB, 
Massachusetts 

Dave  Buckley,  MITRE  Corporation,  Colorado  Springs,  Colorado 

Howard  E.  Carolus,  Lt.  Col. ,  Electronic  Systems  Division 
Field  Office,  Colorado  Springs,  Colorado 

R.L.  Clark,  Hughes  Aircraft  Company,  Fullerton,  California 

Vincent  J.  Cosentino,  GTE  Sylvania,  Inc.,  Needham  Heights, 
Massachusetts 

Bob  Cutter,  Capt, ,  Electronic  Systems  Division,  Hanscom  AFB, 
Massachusetts 

Bill  Dean,  Air  Force  Institute  of  Technology,  Wright-Patterson 
AFB,  Ohio 

Chuck  Denny,  Electronic  Systems  Division,  Hanscom  AFB, 

*  Massachusetts 

Bob  Doane,  Electronic  Systems  Division,  Hanscom  AFB, 

*  Massachusetts 

Bud  Drutz,  System  Development  Corporation,  McLean,  Virginia 

Grace  Dugas,  Capt.,  Electronic  Systems  Division,  Hanscom 
AFB,  Massachusetts 

Jim  Earlain,  MITRE  Corporation,  Bedford,  Massachusetts 
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TABLE  III  (Continued) 


PERSONNEL  RESPONDING  TO  INTERVIEW  QUESTIONNAIRE 


Bill  Fohrman,  Lt.  Col,,  Aeronautical  Systems  Division,  Wright- 
Patterson  AFB,  Ohio 

Bob-  Frye,  MITRE  Corporation,  Bedford,  Massachusetts 

Kurt  Fischer,  Computer  Sciences  Corporation,  Falls  Church, 
Virginia 

Sam  Galzarano,  Electronic  Systems  Division,  Hanscom  AFB, 
Massachusetts 

Bob  Gildea,  MITRE  Corporation,  Bedford,  Massachusetts 

George  Gehlauf,  Air  Force  Logistics  Command  Headquarters, 
Wright-Patterson  AFB,  Ohio 

John  Glore,  MITRE  Corporation,  Bedford,  Massachusetts 

Joseph  S.  Greene,  Jr.,  Lt.  Col.,  Strategic  Air  Command 
Headquarters,  Offutt  AFB,  Nebraska 

Dave  Herrelko,  Capt. ,  Air  Force  Systems  Command  Headquarters, 
Andrews  AFB,  Maryland 

Maretta  Holden,  Boeing  Witcha  Co. /Seattle  Branch,  Seattle, 
Washington 

Bob  Irwin,  MITRE  Corporation,  Colorado  Springs,  Colorado 

Gary  Klayman,  TRW,  Inc.,  Newberry  Park,  California 

A1  Kopp,  Capt. ,  Air  Force  Systems  Command  Headquarters, 

Andrews  AFB,  Maryland 

Roger  Linder,  GTE  Sylvania,  Inc.,  Needham  Heights, 
Massachusetts 

Steve  McCulloch,  Maj.,  Aerospace  Defense  Command  Headquarters, 
Colorado  Springs,  Colorado 

David  R.  McMillan,  MITRE  Corporation,  Bedford,  Massachusetts 

Norm  Michaud,  Col.,  Electronic  Systems  Division,  Hanscom 
AFB,  Massachusetts 

Joe  Mock,  Maj.,  Space  and  Missile  Systems  Division,  Los 
Angeles,  California 
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TABLE  III  (Continued) 


PERSONNEL  RESPONDING  TO  INTERVIEW  QUESTIONNAIRE 

" 

Jack  Munson,  System  Development  Corporation,  Santa  Monica, 
California 

George  Neil,  System  Development  Corporation,  Santa  Monica, 
California 

R.L.  Oldham,  TRV/  Systems,  Inc,,  Rendondo  Beach,  California 

Tony  Pease,  Booz-Allen  and  Hamilton,  Bethesda,  Maryland 

A1  Roberts,  MITRE  Corporation,  Bedford,  Massachusetts 

Anthony  D.  Salvucci,  Electronic  Systems  Division,  Hanscom 
AFB,  Massachusetts 

C.  Schneider,  Air  Force  Plant  Representative  Office/TRW, 
Redondo  Beach,  California 

Tony  Schuman,  TRW  Systems,  Inc,,  Redondo  Beach,  California 

Lloyd  Searle,  System  Development  Corporation,  Santa  Monica, 
California 

Oscar  Shapiro,  MITRE  Corporation,  Bedford,  Massachusetts 

Henry  B,  Stelling,  Gen,,  Electronic  Systems  Division, 

Hanscom  AFB,  Massachusetts 

Richard  J.  Sylvester,  Aeronautical  Systems  Division,  Wright- 
Patterson  AFB,  Ohio 

George  Trever,  Capt. ,  Air  Force  Plant  Representative  Office/ 
TRW,  Redondo  3each,  California 

Bill  White,  MITRE  Corporation,  Bedford,  Massachusetts 
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Interview  Questions.  Vari 
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TABLE  IV 


QUESTIONS  RESULTING  IN  QUANTIFIABLE  DATA 


Question 
(Section  I) 

Variable 

Short  Title 

Classifi¬ 
cation  of 
Data 

1 

Q1 

Level  of  Assignment 

Nominal 

♦ 

2 

Q2 

Function  Assigned  to 

Nominal 

SI 

3 

Q3 

Grade  Level 

Nominal 

0 

4 

Q4 

Education  Level 

Ordinal 

% 

5 

Q5 

Q6 

Q7 

Q8 

Formal  Training  in« 

System  Engineering 
Software  Engineering 
Hardware  Engineering 
Acquisition  Management 

Interval 

Interval 

Interval 

Interval 

6 

Q9 

Years  in  Systems  Dev./Acq. 

Interval 

7 

Q10 

Years  in  Dev./Acq.  of 
Software 

Interval 

K 

1 

I  f 

8 

Qll 

How  recent  was  Experience? 

Interval  . 

\  | 

• 

[  • 

9 

Q12 

Q13 

Ql4 

Q15 

Ql6 

Q17 

Ql8 

Type  of  Software  Appli¬ 
cation  Experience* 

Airborne  Systems 
Intelligence  Systems 

C3  Systems 

Space  and  Missile 

Systems 

Scientific  Systems 
Business  System 

Other  Types  of  Systems 

Interval 

Interval 

Interval 

Interval 

Interval 

Interval 

Interval 

i 

« 

10 

Q19 

Q20 

Q21 

Q22 

Q2  3 

Q24 

Type  of  Software  Ex¬ 
perience  i 

Programming 

Systems  Analysis 
Maintenance 

Test  and  Evaluation 
Engineering 

System  Design 

Interval 

Interval 

Interval 

Interval 

Interval 

Interval 

TABLE  IV  (Continued) 


QUESTIONS  RESULTING  IN  QUANTIFIABLE  DATA 


Question 

(Sections 

I  &  II) 

Variable 

Short  Title 

Classifi¬ 
cation  of 
Data 

Q25 

Q26 

Management 

Other  Experiences 

Interval 

Interval 

11 

Q27 

Number  System  Acq.  Pro¬ 
grams 

Interval 

3 

Q28 

Years  Assigned  to  the 
Program 

Interval 

7 

Q29 

Q30 

Q31 

Q32 

Q33 

Program  Characteristics* 
Total  Development  Cost 
Number  of  CPC  Is 

Number  Lines  of  Code 

Cost  Ratio-Software 
to  Hardware 

Percent  Contract 

Comple te 

Ordinal 

Ordinal 

Ordinal 

Interval 

Interval 

8* 

Q34 

CPC  Is  Defined  at  Right 

Size? 

Interval 

9* 

Q35 

Familiar  with  Past 
Criteria? 

Interval 

12* 

Q36 

Criteria  Used  Different 
from  Past? 

Interval 

14* 

Q37 

Any  Other  Criteria  Should 

Be  Used? 

Interval 

15* 

Q38 

Criteria  for  CPC  Is  Dif¬ 
ferent  from  CIs? 

Interval 

16* 

Q39 

Experienced  Problems  with 
CPC I  Structure? 

Interval 

17* 

Q40 

Alternative  CPC I  Structure 
Evaluated  ? 

Interval 

2 


r 


TABLE  IV  (Continued) 


QUESTIONS  RESULTING  IN  QUANTIFIABLE  DATA 


Question 

Classifi¬ 

(Sections 

cation  cf 

II  &  III) 

Variable 

Short  Title 

Data 

19* 

Q4l 

Suggestions  for  Improving 

System  Requirements  Decom¬ 

position  Process 

Interval 

1 

Q42 

Horizontal  Allocation 

Feasible? 

Interval 

2 

Effectiveness  of  Horizon¬ 

tal  Allocation  ini 

q43 

Reducing  Development 

Cost 

Interval 

q44 

Reducing  Development 

Time 

Interval 

Q45 

Increasing  Visibility 

to  Management 

Interval 

Q46 

Improve  Software 

Quality 

Interval 

047 

More  Efficient  De¬ 

bugging  and  Testing 

Interval 

Q48 

Increasing  Visibility 

to  User 

Interval 

Q49 

Making  Software  Main¬ 

tenance  Easier  and  Less 

Costly 

Interval 

Q50 

Increasing  Morale 

Interval 

Q51 

Reducing  Software 

Integration  Task 

Interval 

Q52 

Highlighting  Problems 

Earlier 

Interval 

Q53 

Reducing  Complexity  in 

Decomposition  Process 

Interval 

Q54 

Improving  Training 

Effectiveness 

Interval 

3* 

Q55 

Adversely  Impact  Contractor 

Interval 
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TABLE  IV  (Continued) 


QUESTIONS  RESULTING  IN  QUANTIFIABLE  DATA 


Question 

Classifi¬ 

(Section 

cation  of 

III) 

Variable 

Short  Title 

Data 

4* 

Q56 

Other  Advantages/ 
Disadvantages 

Interval 

5 

Q57 

Should  Horizontal  Alloca¬ 
tion  Be  Implemented? 

Interval 

*  In  addition  to  soliciting  quantifiable  answers,  these 
questions  asked  the  respondent  to  explain,  expand  on  or 
provide  rationale  for  his  answer. 


TABLE  V 


QUESTIONS  RESULTING  IN  QUALITATIVE  DATA 

Question 

(Section  II)  Short  Title 

1  Program  ID 

2  Program  Description 

4  Job  Description 

5  Time  of  System  Requirements  Allocation 

6  Who  Performed  CPC  I  Selection? 

8*  CPCIs  Defined  at  Right  Size? 

9*  Familiar  with  Past  Criteria? 

10  How  Were  Criteria  Selected? 

11  Advantages/Disadvantages  of  the  Criteria 

12*  Criteria  Used  Different  From  Past? 

13  Advantages/Disadvantages  of  Past  Criteria 

14*  Any  Other  Criteria  Should  Be  Used? 

15*  Criteria  for  CPCIs  Different  From  CIs? 

l6*  Experienced  Problems  with  CPCI  Structure? 

17*  Alternative  CPCI  Structure  Evaluated? 

18  Recommended  Changes  in  CPCI  Selection  Process 

19*  Suggestions  for  Improving  System  Requirements 

Decomposition  Process 


216 


TABLE  V  (Continued) 


QUESTIONS  RESULTING  IN  QUALITATIVE  DATA 

Question 

(Section  III)  Short  Title 

3*  Adversely  Impact  Contractor 

4*  Other  Advantages/Disadvantages 

5*  Should  Horizontal  Allocation  Be  Implemented? 


*  These  questions  asked  the  respondent  to  provide  qualita¬ 
tive  support  for  his  dichotomous  yes/no  answers. 
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Tables  of  Bivariate  Correlations 
These  tables  present  the  significant  linear  correlations 
between  each  pair  of  variables  defined  in  Appendix  F.  The 
sign  indicates  either  a  negative  or  positive  relationship 
between  the  variables  as  coded  in  Appendices  C,  H,  and  I. 
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TABLE  XVIII 


SYSTEM  DE VELOFKENT/ AC QU IS IT ION  PROGRAMS 
REPRESENTED  IN  THE  INTERVIEWS 


Program  ID  Type  of  System 


AEGIS 
AFEES 
ALQ  131 
ATEC 

AW  ACS 
B-52  OAS 
BETA 


BMEWS  TOR 
CCPDS 
CONUS  OTH 
DMSP 
DSP 

F-15  AISF 
GEADGE 
GEODSS 
GLCM 

ICAM 


JSS 

JTIDS 

OASIS 
PAVE  PAWS 
Project  Impact 
SAC DIN 
SAGE 


Fleet  Air  Defense  System 

Air  Force  Entrance  Examination  System 

ECM  Pod  For  Aeronautical  Systems 

Ground-based  Communications  Monitoring 
System 

Airborne  Warning  and  Control  System 

Offensive  Avionics  System 

Tactical  Intelligence  Data  System 

Ballistic  Missile  Defense  System 
Technology  Program 

Tactical  Operations  Room  Upgrade 

Command  and  Control  System 

Over  the  Horizon  Radar  System 

Defense  Meteorological  Satellite  Program 

Command  and  Control  System 

Avionics  Integrated  Support  Facility 

4l2L  System  Replacement 

Deep  Space  Tracking  System 

Ground  Launched  Cruise  Missile 

Industry  Manufacturing  Productivity 
Program 

Command  and  Control  System 

Command,  Control  and  Communications  System 

NASA  V.king  Program 

Intelligence  Processing  System 

Command  and  Control  System 

Office  Automation  Program 

Communications  System 

Air  Defense  System 

SAMSO  Satellite  Control  Facility  System 
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TABLE  XVIII  (Continued) 


SYSTEM  DEVELOPMENT/AC  QU IS IT ION  PROGRAMS 
REPRESENTED  IN  THE  INTERVIEWS 


Program  ID 

Type  of  System 

SEEK  IGLOO 

Ground-based  Radar  System 

SPADOC 

Space  Defense  Operations  Center  System 

S UR TASS 

Sonar  Surveillance  System 

STEM  (407L) 

System  Training  and  Exercise  Module 

TAC  FIRE 

Field  Artillery  Fire  Detection  Center 
System 

TAC  STADS 

Joint  Tactical  Air  Defense  System 

TCCF  (478T) 

Tactical  Communication  Control  Facility 

TIPI 

Intelligence  Display  Control  Storage  and 
Retrieval  System 

— 

T03  Follow-on  System 

32  9A 

A- 10  Aircraft  System 

MAC  IMS  (4l6L) 

Passenger  Service  and  Cargo  Automation 
Program 

427M 

Command  and  Control  Upgrade  Program 

436M 

WWMCCS  Interface  Program 
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SIGNIFICANCE  TESTS 
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Result 

Q39 

36 

.722 

2.664 

1.645 

Reject 

Q42 

39 

.821 

4.010 

1.645 

Reject 

t 

Q43 

38 

.711 

2.601 

1.645 

Reject 

• 

q44 

38 

.710 

2.589 

1.645 

Reject 

* 

Q45 

38 

.815 

3.884 

1.645 

Reject 

i  • 

Q46 

38 

.789 

3.563 

1.645 

Reject 

Q47 

38 

.816 

3.896 

1.645 

Reject 

Q48 

38 

.842 

4.216 

1.645 

Reject 

Q49 

3? 

.702 

2.457 

1.645 

Reject 

Q50 

3? 

.675 

2.129 

1.645 

Re ject 

i 

Q51 

37 

.784 

3.455 

1.645 

Reject 

\ 

i 

Q52 

37 

.784 

3.455 

1.645 

Reject 

Q53 

38 

.605 

1.295 

1.645 

Fail  to  Reject 

Q54 

38 

.868 

4.537 

1.645 

Reject 

Q55 

4o 

.400 

-0.791 

1.645 

Fail  to  Reject 

('  i 

Q57 

37 

.757 

3.126 

1.645 

Reject 

* 

% 

j 

*  The 
<Ho 

students*  one-tailed  t  test  was  used  to  test  the  null 
)  and  alternate  (H  )  hypotheses  described  below i 
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Po=0.50 

V 

Po>0.50 

Test 

Statistic i 

V'-V 


Significance  Level «  a=.05 

Degrees  of  Freedora(d. f . ) i  n-1 
Rejection  Region i  t>tfl  d  f 


where 


a.  PQis  equal  to  the  percentage  of  respondents 

who  answered  a  question  positively  (i.e.,  yes 
for  a  dichotomous  question,  and  at  least  some¬ 
what  effective  for  the  perceived  effectiveness 
questions ) . 

b.  n  is  the  number  of  respondents  answering  that 
question. 

c.  Fail  to  Reject  means  that  there  is  sufficient 
evidence  to  support  the  null  hypothesis  that 
P  is  equal  to  0.50. 

d.  Reject  means  that  the  evidence  supports  the 
alternate  hypothesis  that  P  is  greater  than 

0.50. 


265 


♦ 


vita 


Virgil  L.  Cooper  was  bom  on  8  December  1946  in 
Jamestown,  Tennessee,  He  graduated  from  the  Alvin  C,  York 
Agricultural  Institute  in  1964,  and  enlisted  in  the  U.S. 

Air  Force  in  April  1966,  After  completing  the  Airborne 
Radio  Repairman  Course  at  the  Keesler  Technical  Training 
Center,  Keesler  AFB,  Mississippi,  he  was  assigned  to  the  Air 
Force  Eastern  Test  Range  at  Patrick  AFB,  Florida  in  January 
1967.  During  the  next  years,  he  served  as  a  High  Fre¬ 
quency  Operator/Technician  on  the  Apollo  Range  Instrumenta¬ 
tion  Aircraft  (EC-135N). 

In  August  1972,  Staff  Sergeant  Cooper  was  selected  for 
the  Airman  Education  and  Commissioning  Program,  and  trans¬ 
ferred  to  Oklahoma  State  University  where  he  received  a 
Bachelor  of  Science  degree  in  Mathematics  in  May  1974,  After 
completing  Officers  Training  School  in  August  1974,  he  was 
commissioned  a  Second  Lieutenant  and  assigned  to  the  Elec¬ 
tronic  Systems  Division (ESD)  of  the  Air  Force  Systems  Command 
at  Hanscom  AFB,  Massachusetts  as  a  Computer  Systems  Analyst, 
After  9  months,  he  was  transferred  to  the  ESD  Field  Office  in 
Colorado  Springs,  Colorado.  During  the  next  3  years,  he 
participated  in  all  phases  of  the  system  acquisition  cycle 
as  a  Computer  Systems  Analyst,  and  System  Integration  and 
Test  Monitor  on  a  major  C^  acquisition  program. 

In  May  1978,  he  entered  the  Air  Force  Institute  of 
Technology  as  a  graduate  student  in  Systems  Management. 

Permanent  Address  1  Route  3,  Box  361 

Jamestown,  Tennessee  3^556 


266 


